[ 506.726053] nova-conductor[53390]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. [ 507.771065] nova-conductor[53390]: INFO dbcounter [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Registered counter for database nova_api [ 507.780065] nova-conductor[53390]: DEBUG oslo_db.sqlalchemy.engines [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=53390) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 507.783377] nova-conductor[53390]: DEBUG dbcounter [-] [53390] Writer thread running {{(pid=53390) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 507.806418] nova-conductor[53390]: DEBUG nova.context [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),3368e979-e585-49c6-913f-77bff95fda6f(cell1) {{(pid=53390) load_cells /opt/stack/nova/nova/context.py:464}} [ 507.808144] nova-conductor[53390]: DEBUG oslo_concurrency.lockutils [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53390) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 507.808360] nova-conductor[53390]: DEBUG oslo_concurrency.lockutils [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53390) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 507.808798] nova-conductor[53390]: DEBUG oslo_concurrency.lockutils [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53390) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 507.809215] nova-conductor[53390]: DEBUG oslo_concurrency.lockutils [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53390) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 507.809416] nova-conductor[53390]: DEBUG oslo_concurrency.lockutils [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53390) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 507.810339] nova-conductor[53390]: DEBUG oslo_concurrency.lockutils [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53390) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 507.811655] nova-conductor[53390]: INFO dbcounter [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Registered counter for database nova_cell0 [ 507.813889] nova-conductor[53390]: INFO dbcounter [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Registered counter for database nova_cell1 [ 507.816484] nova-conductor[53390]: DEBUG oslo_db.sqlalchemy.engines [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=53390) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 507.816950] nova-conductor[53390]: DEBUG oslo_db.sqlalchemy.engines [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=53390) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 507.819913] nova-conductor[53390]: DEBUG dbcounter [-] [53390] Writer thread running {{(pid=53390) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 507.820082] nova-conductor[53390]: DEBUG dbcounter [-] [53390] Writer thread running {{(pid=53390) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 507.876244] nova-conductor[53390]: DEBUG oslo_concurrency.lockutils [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Acquiring lock "singleton_lock" {{(pid=53390) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}} [ 507.876418] nova-conductor[53390]: DEBUG oslo_concurrency.lockutils [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Acquired lock "singleton_lock" {{(pid=53390) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}} [ 507.876656] nova-conductor[53390]: DEBUG oslo_concurrency.lockutils [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Releasing lock "singleton_lock" {{(pid=53390) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}} [ 507.877090] nova-conductor[53390]: INFO oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Starting 2 workers [ 507.881315] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Started child 53809 {{(pid=53390) _start_child /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/service.py:575}} [ 507.884947] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Started child 53810 {{(pid=53390) _start_child /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/service.py:575}} [ 507.885623] nova-conductor[53809]: INFO nova.service [-] Starting conductor node (version 0.1.0) [ 507.886007] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Full set of CONF: {{(pid=53390) wait /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/service.py:649}} [ 507.886206] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ******************************************************************************** {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2589}} [ 507.886356] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] Configuration options gathered from: {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2590}} [ 507.886543] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] command line args: ['--config-file', '/etc/nova/nova.conf'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2591}} [ 507.886872] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] config files: ['/etc/nova/nova.conf'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2592}} [ 507.887033] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ================================================================================ {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2594}} [ 507.887438] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] allow_resize_to_same_host = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.887683] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] arq_binding_timeout = 300 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.887884] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] block_device_allocate_retries = 60 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.888083] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] block_device_allocate_retries_interval = 3 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.888393] nova-conductor[53810]: INFO nova.service [-] Starting conductor node (version 0.1.0) [ 507.888583] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cert = self.pem {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.888757] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] compute_driver = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.889016] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] compute_monitors = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.889294] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] config_dir = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.889514] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] config_drive_format = iso9660 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.889676] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] config_file = ['/etc/nova/nova.conf'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.889871] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] config_source = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.890074] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] console_host = devstack {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.890283] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] control_exchange = nova {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.890481] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cpu_allocation_ratio = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.890678] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] daemon = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.890881] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] debug = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.891075] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] default_access_ip_network_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.891325] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] default_availability_zone = nova {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.891469] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] default_ephemeral_format = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.891666] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] default_green_pool_size = 1000 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.891964] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.892183] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] default_schedule_zone = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.892377] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] disk_allocation_ratio = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.892572] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] enable_new_services = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.892937] nova-conductor[53809]: INFO dbcounter [None req-c2870c51-b1f7-4e17-9a68-1dfc42bc6f85 None None] Registered counter for database nova_cell1 [ 507.893105] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] enabled_apis = ['osapi_compute'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.893334] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] enabled_ssl_apis = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.893540] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] flat_injected = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.893827] nova-conductor[53810]: INFO dbcounter [None req-e2e3ab71-57f1-4947-97c9-b9874deb5480 None None] Registered counter for database nova_cell1 [ 507.893974] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] force_config_drive = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.894187] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] force_raw_images = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.894384] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] graceful_shutdown_timeout = 5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.894570] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] heal_instance_info_cache_interval = 60 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.894982] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] host = devstack {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.895232] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] initial_cpu_allocation_ratio = 4.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.895442] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] initial_disk_allocation_ratio = 1.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.895631] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] initial_ram_allocation_ratio = 1.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.895884] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.896067] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] instance_build_timeout = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.896239] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] instance_delete_interval = 300 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.896427] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] instance_format = [instance: %(uuid)s] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.896667] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] instance_name_template = instance-%08x {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.896871] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] instance_usage_audit = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.896972] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] instance_usage_audit_period = month {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.897153] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] instance_uuid_format = [instance: %(uuid)s] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.897341] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] instances_path = /opt/stack/data/nova/instances {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.897499] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] internal_service_availability_zone = internal {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.897653] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] key = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.897830] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] live_migration_retry_count = 30 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.898017] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] log_config_append = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.898206] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.898392] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] log_dir = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.898571] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] log_file = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.898704] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] log_options = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.898987] nova-conductor[53809]: DEBUG oslo_db.sqlalchemy.engines [None req-c2870c51-b1f7-4e17-9a68-1dfc42bc6f85 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=53809) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 507.899234] nova-conductor[53810]: DEBUG oslo_db.sqlalchemy.engines [None req-e2e3ab71-57f1-4947-97c9-b9874deb5480 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=53810) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 507.899384] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] log_rotate_interval = 1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.899613] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] log_rotate_interval_type = days {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.899811] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] log_rotation_type = none {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.899962] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.900123] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.900315] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.900520] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.900656] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.900857] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] long_rpc_timeout = 1800 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.901048] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] max_concurrent_builds = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.901233] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] max_concurrent_live_migrations = 1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.901397] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] max_concurrent_snapshots = 5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.901577] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] max_local_block_devices = 3 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.901741] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] max_logfile_count = 30 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.901923] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] max_logfile_size_mb = 200 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.902099] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] maximum_instance_delete_attempts = 5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.902302] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] metadata_listen = 0.0.0.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.902524] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] metadata_listen_port = 8775 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.902703] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] metadata_workers = 2 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.902868] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] migrate_max_retries = -1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.903053] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] mkisofs_cmd = genisoimage {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.903309] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] my_block_storage_ip = 10.180.1.21 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.903503] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writer thread running {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 507.903640] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] my_ip = 10.180.1.21 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.903822] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] network_allocate_retries = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.904016] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.904200] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] osapi_compute_listen = 0.0.0.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.904377] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] osapi_compute_listen_port = 8774 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.904551] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] osapi_compute_unique_server_name_scope = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.904742] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] osapi_compute_workers = 2 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.904906] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] password_length = 12 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.905082] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] periodic_enable = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.905246] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] periodic_fuzzy_delay = 60 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.905423] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] pointer_model = usbtablet {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.905619] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] preallocate_images = none {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.905798] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] publish_errors = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.905929] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] pybasedir = /opt/stack/nova {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.906097] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ram_allocation_ratio = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.906258] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] rate_limit_burst = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.906451] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] rate_limit_except_level = CRITICAL {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.906615] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] rate_limit_interval = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.906776] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] reboot_timeout = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.906938] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] reclaim_instance_interval = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.907126] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] record = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.907290] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] reimage_timeout_per_gb = 20 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.907461] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] report_interval = 120 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.907620] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] rescue_timeout = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.907806] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] reserved_host_cpus = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.907967] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] reserved_host_disk_mb = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.908164] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] reserved_host_memory_mb = 512 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.908338] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] reserved_huge_pages = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.908502] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] resize_confirm_window = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.908661] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] resize_fs_using_block_device = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.908817] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] resume_guests_state_on_host_boot = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.908987] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.909168] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] rpc_response_timeout = 60 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.909329] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] run_external_periodic_tasks = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.909522] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] running_deleted_instance_action = reap {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.909704] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] running_deleted_instance_poll_interval = 1800 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.909899] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writer thread running {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 507.910057] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] running_deleted_instance_timeout = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.910241] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] scheduler_instance_sync_interval = 120 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.910419] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] service_down_time = 720 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.910624] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] servicegroup_driver = db {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.910793] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] shelved_offload_time = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.910957] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] shelved_poll_interval = 3600 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.911145] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] shutdown_timeout = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.911330] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] source_is_ipv6 = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.911489] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ssl_only = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.911658] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] state_path = /opt/stack/data/nova {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.911818] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] sync_power_state_interval = 600 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.911979] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] sync_power_state_pool_size = 1000 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.912214] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] syslog_log_facility = LOG_USER {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.912395] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] tempdir = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.912578] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] timeout_nbd = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.912751] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] transport_url = **** {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.912912] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] update_resources_interval = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.913105] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] use_cow_images = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.913319] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] use_eventlog = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.913493] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] use_journal = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.913655] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] use_json = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.913817] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] use_rootwrap_daemon = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.913981] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] use_stderr = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.914158] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] use_syslog = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.914319] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vcpu_pin_set = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.914493] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vif_plugging_is_fatal = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.914666] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vif_plugging_timeout = 300 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.914877] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] virt_mkfs = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.915053] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] volume_usage_poll_interval = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.915240] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] watch_log_file = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.915490] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] web = /usr/share/spice-html5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 507.915733] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_concurrency.disable_process_locking = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.915947] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.916161] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.916350] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.916527] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_metrics.metrics_process_name = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.916697] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.916866] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.917100] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.auth_strategy = keystone {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.917299] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.compute_link_prefix = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.917505] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.917680] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.dhcp_domain = novalocal {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.917857] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.enable_instance_password = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.918030] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.glance_link_prefix = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.918201] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.instance_list_cells_batch_fixed_size = 100 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.918394] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.instance_list_cells_batch_strategy = distributed {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.918566] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.instance_list_per_project_cells = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.918731] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.list_records_by_skipping_down_cells = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.918897] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.local_metadata_per_cell = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.919075] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.max_limit = 1000 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.919247] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.metadata_cache_expiration = 15 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.919460] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.neutron_default_tenant_id = default {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.919639] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.use_forwarded_for = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.919809] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.use_neutron_default_nets = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.919986] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.vendordata_dynamic_connect_timeout = 5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.920196] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.vendordata_dynamic_failure_fatal = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.920387] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.vendordata_dynamic_read_timeout = 5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.920559] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.vendordata_dynamic_ssl_certfile = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.920732] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.vendordata_dynamic_targets = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.920901] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.vendordata_jsonfile_path = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.921107] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api.vendordata_providers = ['StaticJSON'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.921386] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.backend = dogpile.cache.memcached {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.921572] nova-conductor[53810]: DEBUG nova.service [None req-e2e3ab71-57f1-4947-97c9-b9874deb5480 None None] Creating RPC server for service conductor {{(pid=53810) start /opt/stack/nova/nova/service.py:182}} [ 507.921729] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.backend_argument = **** {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.921999] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.config_prefix = cache.oslo {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.922364] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.dead_timeout = 60.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.922569] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.debug_cache_backend = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.922777] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.enable_retry_client = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.922951] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.enable_socket_keepalive = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.923150] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.enabled = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.923348] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.expiration_time = 600 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.923524] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.hashclient_retry_attempts = 2 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.923758] nova-conductor[53809]: DEBUG nova.service [None req-c2870c51-b1f7-4e17-9a68-1dfc42bc6f85 None None] Creating RPC server for service conductor {{(pid=53809) start /opt/stack/nova/nova/service.py:182}} [ 507.923913] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.hashclient_retry_delay = 1.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.924110] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.memcache_dead_retry = 300 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.924289] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.memcache_password = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.924459] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.924624] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.924786] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.memcache_pool_maxsize = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.924954] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.memcache_pool_unused_timeout = 60 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.925130] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.memcache_sasl_enabled = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.925320] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.memcache_servers = ['localhost:11211'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.925488] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.memcache_socket_timeout = 1.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.925665] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.memcache_username = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.925846] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.proxies = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.926036] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.retry_attempts = 2 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.926194] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.retry_delay = 0.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.926358] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.socket_keepalive_count = 1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.926522] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.socket_keepalive_idle = 1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.926682] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.socket_keepalive_interval = 1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.926846] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.tls_allowed_ciphers = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.927012] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.tls_cafile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.927178] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.tls_certfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.927340] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.tls_enabled = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.927503] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cache.tls_keyfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.927718] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cinder.auth_section = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.927917] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cinder.auth_type = password {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.928108] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cinder.cafile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.928310] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cinder.catalog_info = volumev3::publicURL {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.928475] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cinder.certfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.928640] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cinder.collect_timing = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.928826] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cinder.cross_az_attach = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.928992] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cinder.debug = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.929169] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cinder.endpoint_template = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.929335] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cinder.http_retries = 3 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.929498] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cinder.insecure = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.929664] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cinder.keyfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.929832] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cinder.os_region_name = RegionOne {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.929994] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cinder.split_loggers = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.930176] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cinder.timeout = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.930351] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] compute.consecutive_build_service_disable_threshold = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.930514] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] compute.cpu_dedicated_set = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.930673] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] compute.cpu_shared_set = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.930837] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] compute.image_type_exclude_list = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.930999] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] compute.live_migration_wait_for_vif_plug = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.931181] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] compute.max_concurrent_disk_ops = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.931342] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] compute.max_disk_devices_to_attach = -1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.931505] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.931673] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.931842] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] compute.resource_provider_association_refresh = 300 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.932009] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] compute.shutdown_retry_interval = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.932229] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] compute.vmdk_allowed_types = ['streamOptimized', 'monolithicSparse'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.932443] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] conductor.workers = 2 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.932632] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] console.allowed_origins = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.932798] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] console.ssl_ciphers = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.932970] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] console.ssl_minimum_version = default {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.933210] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] consoleauth.token_ttl = 600 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.933443] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.cafile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.933611] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.certfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.933785] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.collect_timing = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.933947] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.connect_retries = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.934119] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.connect_retry_delay = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.934298] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.endpoint_override = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.934447] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.insecure = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.934608] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.keyfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.934767] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.max_version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.934924] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.min_version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.935093] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.region_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.935290] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.service_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.935474] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.service_type = accelerator {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.935637] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.split_loggers = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.935828] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.status_code_retries = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.935976] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.status_code_retry_delay = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.936148] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.timeout = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.936337] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.936499] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] cyborg.version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.936692] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.backend = sqlalchemy {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.936876] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.connection = **** {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.937062] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.connection_debug = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.937238] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.connection_parameters = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.937408] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.connection_recycle_time = 3600 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.937576] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.connection_trace = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.937741] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.db_inc_retry_interval = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.937906] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.db_max_retries = 20 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.938080] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.db_max_retry_interval = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.938267] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.db_retry_interval = 1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.938452] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.max_overflow = 50 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.938621] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.max_pool_size = 5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.938789] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.max_retries = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.938965] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.mysql_sql_mode = TRADITIONAL {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.939141] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.mysql_wsrep_sync_wait = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.939306] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.pool_timeout = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.939478] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.retry_interval = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.939636] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.slave_connection = **** {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.939803] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.sqlite_synchronous = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.939971] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] database.use_db_reconnect = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.940184] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.backend = sqlalchemy {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.940382] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.connection = **** {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.940553] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.connection_debug = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.940723] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.connection_parameters = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.940919] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.connection_recycle_time = 3600 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.941098] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.connection_trace = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.941289] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.db_inc_retry_interval = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.941471] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.db_max_retries = 20 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.941672] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.db_max_retry_interval = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.941810] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.db_retry_interval = 1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.941979] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.max_overflow = 50 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.942167] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.max_pool_size = 5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.942347] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.max_retries = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.942520] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.mysql_sql_mode = TRADITIONAL {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.942680] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.mysql_wsrep_sync_wait = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.942844] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.pool_timeout = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.943029] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.retry_interval = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.943185] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.slave_connection = **** {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.943354] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] api_database.sqlite_synchronous = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.943555] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] devices.enabled_mdev_types = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.943763] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.943932] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ephemeral_storage_encryption.enabled = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.944110] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ephemeral_storage_encryption.key_size = 512 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.944336] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.api_servers = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.944519] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.cafile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.944703] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.certfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.944871] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.collect_timing = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.945064] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.connect_retries = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.945234] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.connect_retry_delay = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.945420] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.debug = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.945634] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.default_trusted_certificate_ids = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.945803] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.enable_certificate_validation = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.945993] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.enable_rbd_download = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.946135] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.endpoint_override = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.946306] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.insecure = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.946473] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.keyfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.946631] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.max_version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.946791] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.min_version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.946953] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.num_retries = 3 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.947137] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.rbd_ceph_conf = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.947346] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.rbd_connect_timeout = 5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.947541] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.rbd_pool = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.947711] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.rbd_user = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.947871] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.region_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.948064] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.service_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.948240] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.service_type = image {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.948407] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.split_loggers = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.948565] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.status_code_retries = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.948722] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.status_code_retry_delay = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.948881] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.timeout = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.949077] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.valid_interfaces = ['internal', 'public'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.949254] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.verify_glance_signatures = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.949416] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] glance.version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.949899] nova-conductor[53810]: DEBUG nova.service [None req-e2e3ab71-57f1-4947-97c9-b9874deb5480 None None] Join ServiceGroup membership for this service conductor {{(pid=53810) start /opt/stack/nova/nova/service.py:199}} [ 507.950095] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] guestfs.debug = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.950347] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.config_drive_cdrom = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.950581] nova-conductor[53810]: DEBUG nova.servicegroup.drivers.db [None req-e2e3ab71-57f1-4947-97c9-b9874deb5480 None None] DB_Driver: join new ServiceGroup member devstack to the conductor group, service = {{(pid=53810) join /opt/stack/nova/nova/servicegroup/drivers/db.py:44}} [ 507.950733] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.config_drive_inject_password = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.950925] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.dynamic_memory_ratio = 1.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.951115] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.enable_instance_metrics_collection = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.951288] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.enable_remotefx = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.951619] nova-conductor[53809]: DEBUG nova.service [None req-c2870c51-b1f7-4e17-9a68-1dfc42bc6f85 None None] Join ServiceGroup membership for this service conductor {{(pid=53809) start /opt/stack/nova/nova/service.py:199}} [ 507.951767] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.instances_path_share = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.951934] nova-conductor[53809]: DEBUG nova.servicegroup.drivers.db [None req-c2870c51-b1f7-4e17-9a68-1dfc42bc6f85 None None] DB_Driver: join new ServiceGroup member devstack to the conductor group, service = {{(pid=53809) join /opt/stack/nova/nova/servicegroup/drivers/db.py:44}} [ 507.952091] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.iscsi_initiator_list = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.952305] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.limit_cpu_features = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.952485] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.mounted_disk_query_retry_count = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.952658] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.952821] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.power_state_check_timeframe = 60 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.952983] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.power_state_event_polling_interval = 2 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.953170] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.953354] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.use_multipath_io = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.953517] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.volume_attach_retry_count = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.953678] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.volume_attach_retry_interval = 5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.953868] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.vswitch_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.954060] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] hyperv.wait_soft_reboot_seconds = 60 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.954292] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] mks.enabled = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.954936] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.955149] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] image_cache.manager_interval = 2400 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.955345] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] image_cache.precache_concurrency = 1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.955547] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] image_cache.remove_unused_base_images = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.955695] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.955867] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.956113] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] image_cache.subdirectory_name = _base {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.956255] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.api_max_retries = 60 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.956427] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.api_retry_interval = 2 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.956588] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.auth_section = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.956773] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.auth_type = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.956935] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.cafile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.957104] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.certfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.957277] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.collect_timing = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.957450] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.conductor_group = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.957614] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.connect_retries = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.957772] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.connect_retry_delay = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.957929] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.endpoint_override = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.958104] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.insecure = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.958265] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.keyfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.958427] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.max_version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.958584] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.min_version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.958747] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.peer_list = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.958911] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.region_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.959086] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.serial_console_state_timeout = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.959250] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.service_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.959444] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.service_type = baremetal {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.959607] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.split_loggers = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.959765] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.status_code_retries = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.959922] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.status_code_retry_delay = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.960110] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.timeout = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.960330] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.valid_interfaces = ['internal', 'public'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.960509] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ironic.version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.960735] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.960935] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] key_manager.fixed_key = **** {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.961172] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.961366] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican.barbican_api_version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.961528] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican.barbican_endpoint = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.961727] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican.barbican_endpoint_type = public {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.961908] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican.barbican_region_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.962101] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican.cafile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.962290] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican.certfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.962494] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican.collect_timing = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.962658] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican.insecure = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.962815] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican.keyfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.962980] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican.number_of_retries = 60 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.963208] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican.retry_delay = 1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.963412] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican.send_service_user_token = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.963609] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican.split_loggers = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.963783] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican.timeout = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.963952] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican.verify_ssl = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.964146] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican.verify_ssl_path = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.964317] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican_service_user.auth_section = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.964480] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican_service_user.auth_type = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.964638] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican_service_user.cafile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.964794] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican_service_user.certfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.964960] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican_service_user.collect_timing = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.965134] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican_service_user.insecure = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.965294] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican_service_user.keyfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.965457] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican_service_user.split_loggers = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.965617] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] barbican_service_user.timeout = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.965810] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vault.approle_role_id = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.965992] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vault.approle_secret_id = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.966278] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vault.cafile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.966340] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vault.certfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.966502] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vault.collect_timing = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.966667] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vault.insecure = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.966825] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vault.keyfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.967032] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vault.kv_mountpoint = secret {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.967198] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vault.kv_path = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.967370] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vault.kv_version = 2 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.967535] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vault.namespace = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.967696] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vault.root_token_id = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.967862] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vault.split_loggers = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.968029] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vault.ssl_ca_crt_file = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.968194] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vault.timeout = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.968362] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vault.use_ssl = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.968551] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vault.vault_url = http://127.0.0.1:8200 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.968753] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.auth_section = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.968913] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.auth_type = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.969082] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.cafile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.969242] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.certfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.969408] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.collect_timing = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.969588] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.connect_retries = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.969747] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.connect_retry_delay = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.969907] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.endpoint_override = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.970084] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.insecure = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.970318] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.keyfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.970508] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.max_version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.970668] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.min_version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.970832] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.region_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.970991] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.service_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.971185] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.service_type = identity {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.971351] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.split_loggers = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.971511] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.status_code_retries = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.971674] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.status_code_retry_delay = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.971829] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.timeout = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.972015] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.valid_interfaces = ['internal', 'public'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.972206] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] keystone.version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.972462] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.connection_uri = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.972647] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.cpu_mode = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.972815] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.cpu_model_extra_flags = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.972983] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.cpu_models = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.973176] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.cpu_power_governor_high = performance {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.973353] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.cpu_power_governor_low = powersave {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.973535] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.cpu_power_management = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.973724] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.cpu_power_management_strategy = cpu_state {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.973915] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.device_detach_attempts = 8 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.974091] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.device_detach_timeout = 20 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.974261] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.disk_cachemodes = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.974424] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.disk_prefix = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.974594] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.enabled_perf_events = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.974758] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.file_backed_memory = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.974920] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.gid_maps = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.975116] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.hw_disk_discard = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.975277] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.hw_machine_type = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.975452] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.images_rbd_ceph_conf = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.975615] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.975799] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.975969] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.images_rbd_glance_store_name = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.976149] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.images_rbd_pool = rbd {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.976439] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.images_type = default {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.976517] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.images_volume_group = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.976698] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.inject_key = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.976862] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.inject_partition = -2 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.977036] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.inject_password = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.977221] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.iscsi_iface = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.977387] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.iser_use_multipath = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.977549] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.live_migration_bandwidth = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.977714] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.live_migration_completion_timeout = 800 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.977880] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.live_migration_downtime = 500 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.978048] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.live_migration_downtime_delay = 75 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.978214] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.live_migration_downtime_steps = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.978376] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.live_migration_inbound_addr = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.978536] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.live_migration_permit_auto_converge = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.978697] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.live_migration_permit_post_copy = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.978864] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.live_migration_scheme = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.979057] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.live_migration_timeout_action = abort {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.979251] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.live_migration_tunnelled = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.979424] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.live_migration_uri = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.979612] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.live_migration_with_native_tls = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.979795] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.max_queues = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.979962] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.mem_stats_period_seconds = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.980145] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.nfs_mount_options = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.980422] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.980595] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.num_aoe_discover_tries = 3 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.980759] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.num_iser_scan_tries = 5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.980926] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.num_memory_encrypted_guests = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.981109] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.num_nvme_discover_tries = 5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.981281] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.num_pcie_ports = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.981448] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.num_volume_scan_tries = 5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.981666] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.pmem_namespaces = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.981831] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.quobyte_client_cfg = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.982096] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.982295] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.rbd_connect_timeout = 5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.982469] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.rbd_destroy_volume_retries = 12 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.982633] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.982793] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.rbd_secret_uuid = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.982953] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.rbd_user = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.983129] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.realtime_scheduler_priority = 1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.983345] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.remote_filesystem_transport = ssh {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.983516] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.rescue_image_id = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.983682] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.rescue_kernel_id = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.983843] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.rescue_ramdisk_id = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.984026] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.rng_dev_path = /dev/urandom {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.984191] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.rx_queue_size = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.984358] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.smbfs_mount_options = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.984591] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.984757] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.snapshot_compression = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.984915] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.snapshot_image_format = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.985145] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.985320] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.sparse_logical_volumes = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.985486] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.swtpm_enabled = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.985680] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.swtpm_group = tss {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.985857] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.swtpm_user = tss {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.986041] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.sysinfo_serial = unique {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.986206] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.tb_cache_size = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.986369] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.tx_queue_size = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.986589] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.uid_maps = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.986695] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.use_virtio_for_bridges = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.986865] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.virt_type = kvm {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.987041] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.volume_clear = zero {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.987209] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.volume_clear_size = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.987383] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.volume_use_multipath = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.987541] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.vzstorage_cache_path = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.987710] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.987877] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.vzstorage_mount_group = qemu {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.988054] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.vzstorage_mount_opts = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.988251] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.vzstorage_mount_perms = 0770 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.988504] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.988679] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.vzstorage_mount_user = stack {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.988846] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] libvirt.wait_soft_reboot_seconds = 120 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.989055] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.auth_section = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.989235] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.auth_type = password {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.989404] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.cafile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.989563] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.certfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.989728] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.collect_timing = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.989907] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.connect_retries = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.990083] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.connect_retry_delay = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.990303] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.default_floating_pool = public {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.990474] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.endpoint_override = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.990643] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.extension_sync_interval = 600 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.990803] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.http_retries = 3 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.990963] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.insecure = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.991137] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.keyfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.991297] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.max_version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.991463] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.metadata_proxy_shared_secret = **** {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.991619] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.min_version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.991785] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.ovs_bridge = br-int {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.991947] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.physnets = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.992159] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.region_name = RegionOne {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.992320] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.service_metadata_proxy = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.992485] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.service_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.992659] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.service_type = network {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.992824] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.split_loggers = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.992986] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.status_code_retries = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.993161] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.status_code_retry_delay = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.993322] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.timeout = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.993503] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.valid_interfaces = ['internal', 'public'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.993673] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] neutron.version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.993841] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] notifications.bdms_in_notifications = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.994021] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] notifications.default_level = INFO {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.994198] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] notifications.notification_format = unversioned {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.994364] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] notifications.notify_on_state_change = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.994542] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.994740] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] pci.alias = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.994928] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] pci.device_spec = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.995104] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] pci.report_in_placement = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.995696] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.auth_section = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.995891] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.auth_type = password {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.996090] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.auth_url = http://10.180.1.21/identity {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.996259] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.cafile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.996420] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.certfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.996586] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.collect_timing = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.996758] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.connect_retries = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.996902] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.connect_retry_delay = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.997082] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.default_domain_id = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.997261] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.default_domain_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.997430] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.domain_id = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.997605] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.domain_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.997767] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.endpoint_override = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.997930] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.insecure = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.998099] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.keyfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.998263] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.max_version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.998419] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.min_version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.998588] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.password = **** {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.998748] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.project_domain_id = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.998915] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.project_domain_name = Default {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.999091] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.project_id = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.999268] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.project_name = service {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.999438] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.region_name = RegionOne {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.999598] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.service_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.999765] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.service_type = placement {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 507.999951] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.split_loggers = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.000129] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.status_code_retries = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.000314] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.status_code_retry_delay = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.000474] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.system_scope = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.000633] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.timeout = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.000790] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.trust_id = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.000951] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.user_domain_id = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.001136] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.user_domain_name = Default {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.001302] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.user_id = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.001471] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.username = placement {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.001652] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.valid_interfaces = ['internal', 'public'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.001812] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] placement.version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.002025] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] quota.cores = 20 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.002228] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] quota.count_usage_from_placement = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.002407] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] quota.driver = nova.quota.DbQuotaDriver {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.002600] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] quota.injected_file_content_bytes = 10240 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.002768] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] quota.injected_file_path_length = 255 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.002931] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] quota.injected_files = 5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.003162] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] quota.instances = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.003280] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] quota.key_pairs = 100 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.003436] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] quota.metadata_items = 128 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.003606] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] quota.ram = 51200 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.003767] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] quota.recheck_quota = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.003932] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] quota.server_group_members = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.004113] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] quota.server_groups = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.004283] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] rdp.enabled = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.004615] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.004829] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.005027] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] scheduler.enable_isolated_aggregate_filtering = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.005223] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] scheduler.image_metadata_prefilter = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.005411] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.005584] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] scheduler.max_attempts = 3 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.005750] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] scheduler.max_placement_results = 1000 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.005940] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] scheduler.placement_aggregate_required_for_tenants = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.006124] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] scheduler.query_placement_for_image_type_support = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.006311] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.006488] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] scheduler.workers = 2 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.006688] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.006916] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.007070] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.007253] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.007420] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.007588] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.007754] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.007965] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.enabled_filters = ['ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.008167] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.host_subset_size = 1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.008341] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.hypervisor_version_weight_multiplier = 1.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.008520] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.image_properties_default_architecture = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.008683] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.008849] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.isolated_hosts = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.009026] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.isolated_images = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.009192] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.max_instances_per_host = 50 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.009377] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.max_io_ops_per_host = 8 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.009558] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.num_instances_weight_multiplier = 0.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.009723] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.pci_in_placement = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.009884] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.010064] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.010255] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.010426] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.010591] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.010752] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.010918] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.track_instance_changes = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.011106] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.011286] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] metrics.required = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.011453] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] metrics.weight_multiplier = 1.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.011619] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] metrics.weight_of_unavailable = -10000.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.011784] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] metrics.weight_setting = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.012104] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.012309] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] serial_console.enabled = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.012515] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] serial_console.port_range = 10000:20000 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.012691] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] serial_console.proxyclient_address = 127.0.0.1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.012863] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] serial_console.serialproxy_host = 0.0.0.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.013067] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] serial_console.serialproxy_port = 6083 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.013262] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] service_user.auth_section = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.013446] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] service_user.auth_type = password {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.013609] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] service_user.cafile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.013769] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] service_user.certfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.013933] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] service_user.collect_timing = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.014108] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] service_user.insecure = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.014270] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] service_user.keyfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.014457] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] service_user.send_service_user_token = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.014637] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] service_user.split_loggers = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.014829] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] service_user.timeout = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.015040] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] spice.agent_enabled = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.015213] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] spice.enabled = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.015564] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.015774] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] spice.html5proxy_host = 0.0.0.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.015951] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] spice.html5proxy_port = 6082 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.016129] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] spice.image_compression = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.016293] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] spice.jpeg_compression = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.016454] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] spice.playback_compression = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.016624] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] spice.server_listen = 127.0.0.1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.016792] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] spice.server_proxyclient_address = 127.0.0.1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.016953] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] spice.streaming_mode = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.017129] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] spice.zlib_compression = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.017324] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] upgrade_levels.baseapi = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.017506] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] upgrade_levels.cert = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.017691] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] upgrade_levels.compute = auto {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.017861] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] upgrade_levels.conductor = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.018033] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] upgrade_levels.scheduler = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.018206] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vendordata_dynamic_auth.auth_section = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.018378] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vendordata_dynamic_auth.auth_type = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.018536] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vendordata_dynamic_auth.cafile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.018695] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vendordata_dynamic_auth.certfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.018857] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vendordata_dynamic_auth.collect_timing = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.019029] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vendordata_dynamic_auth.insecure = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.019193] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vendordata_dynamic_auth.keyfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.019358] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vendordata_dynamic_auth.split_loggers = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.019519] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vendordata_dynamic_auth.timeout = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.019727] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.api_retry_count = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.019889] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.ca_file = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.020083] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.cache_prefix = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.020270] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.cluster_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.020443] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.connection_pool_size = 10 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.020608] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.console_delay_seconds = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.020767] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.datastore_regex = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.020929] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.host_ip = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.021101] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.host_password = **** {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.021267] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.host_port = 443 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.021428] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.host_username = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.021592] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.insecure = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.021749] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.integration_bridge = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.021909] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.maximum_objects = 100 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.022078] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.pbm_default_policy = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.022330] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.pbm_enabled = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.022536] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.pbm_wsdl_location = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.022715] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.022877] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.serial_port_proxy_uri = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.023056] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.serial_port_service_uri = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.023538] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.task_poll_interval = 0.5 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.023538] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.use_linked_clone = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.023605] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.vnc_keymap = en-us {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.023712] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.vnc_port = 5900 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.023878] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vmware.vnc_port_total = 10000 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.024114] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vnc.auth_schemes = ['none'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.024292] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vnc.enabled = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.024633] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.024824] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vnc.novncproxy_host = 0.0.0.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.024995] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vnc.novncproxy_port = 6080 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.025196] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vnc.server_listen = 127.0.0.1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.025373] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.025534] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vnc.vencrypt_ca_certs = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.025694] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vnc.vencrypt_client_cert = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.025850] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] vnc.vencrypt_client_key = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.026079] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.disable_compute_service_check_for_ffu = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.026254] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.disable_deep_image_inspection = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.026420] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.disable_fallback_pcpu_query = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.026585] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.disable_group_policy_check_upcall = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.026749] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.disable_libvirt_livesnapshot = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.026910] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.disable_rootwrap = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.027083] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.enable_numa_live_migration = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.027260] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.enable_qemu_monitor_announce_self = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.027409] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.027570] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.handle_virt_lifecycle_events = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.027747] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.libvirt_disable_apic = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.027897] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.never_download_image_if_on_rbd = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.028069] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.qemu_monitor_announce_self_count = 3 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.028234] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.qemu_monitor_announce_self_interval = 1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.028396] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.028561] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.skip_cpu_compare_at_startup = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.028719] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.skip_cpu_compare_on_dest = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.028880] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.029050] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.skip_reserve_in_use_ironic_nodes = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.029214] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.029381] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.029589] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.029760] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] wsgi.client_socket_timeout = 900 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.029920] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] wsgi.default_pool_size = 1000 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.030116] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] wsgi.keep_alive = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.030301] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] wsgi.max_header_line = 16384 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.030470] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] wsgi.secure_proxy_ssl_header = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.030637] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] wsgi.ssl_ca_file = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.030795] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] wsgi.ssl_cert_file = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.030956] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] wsgi.ssl_key_file = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.031138] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] wsgi.tcp_keepidle = 600 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.031346] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.031526] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] zvm.ca_file = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.031688] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] zvm.cloud_connector_url = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.031920] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.032100] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] zvm.reachable_timeout = 300 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.032358] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_policy.enforce_new_defaults = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.032543] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_policy.enforce_scope = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.032720] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_policy.policy_default_rule = default {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.032901] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_policy.policy_dirs = ['policy.d'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.033092] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_policy.policy_file = policy.yaml {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.033314] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.033491] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.033653] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.033811] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.033979] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.034192] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.034383] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.034600] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] profiler.connection_string = messaging:// {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.034787] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] profiler.enabled = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.034973] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] profiler.es_doc_type = notification {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.035170] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] profiler.es_scroll_size = 10000 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.035344] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] profiler.es_scroll_time = 2m {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.035508] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] profiler.filter_error_trace = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.035675] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] profiler.hmac_keys = **** {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.035841] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] profiler.sentinel_service_name = mymaster {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.036041] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] profiler.socket_timeout = 0.1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.036212] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] profiler.trace_requests = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.036380] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] profiler.trace_sqlalchemy = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.036591] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] profiler_jaeger.process_tags = {} {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.036754] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] profiler_jaeger.service_name_prefix = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.036918] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] profiler_otlp.service_name_prefix = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.037143] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] remote_debug.host = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.037430] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] remote_debug.port = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.037523] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.037678] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.037844] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.038010] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.038211] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.038379] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.038549] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.038714] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.038875] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.039049] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.039229] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.039408] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.039632] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.039808] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.039971] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.040183] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.040375] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.040552] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.040724] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.040889] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.041064] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.041237] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.041403] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.041572] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.041739] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.041912] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.ssl = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.042097] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.ssl_ca_file = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.042298] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.ssl_cert_file = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.042472] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.042646] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.ssl_key_file = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.042817] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_rabbit.ssl_version = {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.043049] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.043227] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_notifications.retry = -1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.043445] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.043626] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_messaging_notifications.transport_url = **** {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.043835] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.auth_section = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.044007] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.auth_type = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.044174] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.cafile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.044334] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.certfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.044539] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.collect_timing = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.044701] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.connect_retries = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.044859] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.connect_retry_delay = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.045039] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.endpoint_id = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.045205] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.endpoint_override = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.045398] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.insecure = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.045558] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.keyfile = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.045715] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.max_version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.047302] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.min_version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.047302] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.region_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.047302] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.service_name = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.047302] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.service_type = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.047302] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.split_loggers = False {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.047302] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.status_code_retries = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.047466] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.status_code_retry_delay = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.047466] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.timeout = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.047466] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.valid_interfaces = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.047466] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_limit.version = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.047566] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_reports.file_event_handler = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.047767] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_reports.file_event_handler_interval = 1 {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.047914] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] oslo_reports.log_dir = None {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}} [ 508.048056] nova-conductor[53390]: DEBUG oslo_service.service [None req-59e8c36e-7ed7-4c7e-adaa-21b506ce634f None None] ******************************************************************************** {{(pid=53390) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 517.805287] nova-conductor[53390]: DEBUG dbcounter [-] [53390] Writing DB stats nova_api:SELECT=3 {{(pid=53390) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 517.874859] nova-conductor[53390]: DEBUG dbcounter [-] [53390] Writing DB stats nova_cell0:SELECT=3 {{(pid=53390) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 517.875500] nova-conductor[53390]: DEBUG dbcounter [-] [53390] Writing DB stats nova_cell1:SELECT=3 {{(pid=53390) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 522.961015] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=10,nova_cell1:INSERT=1,nova_cell1:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 522.961406] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=12,nova_cell1:INSERT=1,nova_cell1:UPDATE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 592.465192] nova-conductor[53809]: DEBUG oslo_db.sqlalchemy.engines [None req-4f2d1b77-828b-4655-9fe9-e832bf11e06b None None] Parent process 53390 forked (53809) with an open database connection, which is being discarded and recreated. {{(pid=53809) checkout /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:438}} [ 592.467532] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writer thread running {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 602.469218] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 603.224466] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=15,nova_cell1:INSERT=1,nova_cell1:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 608.251634] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=15,nova_cell1:INSERT=1,nova_cell1:UPDATE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 614.249755] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 637.041839] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=4 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 637.606967] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Took 0.56 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 637.645519] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 637.645853] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 637.651022] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 637.651022] nova-conductor[53809]: INFO dbcounter [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Registered counter for database nova_cell1 [ 637.654663] nova-conductor[53809]: DEBUG oslo_db.sqlalchemy.engines [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=53809) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 637.657513] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writer thread running {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 637.677810] nova-conductor[53810]: DEBUG oslo_db.sqlalchemy.engines [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Parent process 53390 forked (53810) with an open database connection, which is being discarded and recreated. {{(pid=53810) checkout /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:438}} [ 637.680644] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writer thread running {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 637.683969] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=4 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 637.759041] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 637.759041] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 637.759041] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 637.759488] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 637.759818] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 637.759933] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 637.761316] nova-conductor[53809]: INFO dbcounter [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Registered counter for database nova_cell0 [ 637.774514] nova-conductor[53809]: DEBUG oslo_db.sqlalchemy.engines [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=53809) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 637.779621] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writer thread running {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 637.803420] nova-conductor[53809]: DEBUG nova.quota [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Getting quotas for project 2e934b696b374e718726f1a9bc80553e. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 637.807952] nova-conductor[53809]: DEBUG nova.quota [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Getting quotas for user 582af7dcb49e47948f257d7917764674 and project 2e934b696b374e718726f1a9bc80553e. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 637.814058] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] [instance: 2afc8edb-3331-476a-bda3-4f8071461084] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 637.814836] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 637.814927] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 637.815365] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 637.831805] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] [instance: 2afc8edb-3331-476a-bda3-4f8071461084] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 637.831805] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 637.832674] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 637.832817] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 637.889074] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 637.889510] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 637.892784] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 637.892784] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Acquiring lock "compute-rpcapi-router" {{(pid=53809) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}} [ 637.892784] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Acquired lock "compute-rpcapi-router" {{(pid=53809) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}} [ 637.892784] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-599a65a4-4f73-42b2-9d0f-cf0ddb1776b6 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 637.893137] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-599a65a4-4f73-42b2-9d0f-cf0ddb1776b6 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 637.893137] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-599a65a4-4f73-42b2-9d0f-cf0ddb1776b6 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 637.893137] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-599a65a4-4f73-42b2-9d0f-cf0ddb1776b6 None None] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 637.893137] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-599a65a4-4f73-42b2-9d0f-cf0ddb1776b6 None None] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 637.893285] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-599a65a4-4f73-42b2-9d0f-cf0ddb1776b6 None None] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 637.904366] nova-conductor[53809]: INFO nova.compute.rpcapi [None req-599a65a4-4f73-42b2-9d0f-cf0ddb1776b6 None None] Automatically selected compute RPC version 6.2 from minimum service version 66 [ 637.904717] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-599a65a4-4f73-42b2-9d0f-cf0ddb1776b6 None None] Releasing lock "compute-rpcapi-router" {{(pid=53809) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}} [ 638.030490] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Took 0.34 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 638.070075] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 638.070404] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 638.071987] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 638.074136] nova-conductor[53810]: INFO dbcounter [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Registered counter for database nova_cell1 [ 638.079395] nova-conductor[53810]: DEBUG oslo_db.sqlalchemy.engines [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=53810) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 638.082094] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writer thread running {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 638.153644] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 638.154228] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 638.155057] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 638.155604] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 638.155920] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 638.156232] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 638.157743] nova-conductor[53810]: INFO dbcounter [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Registered counter for database nova_cell0 [ 638.166497] nova-conductor[53810]: DEBUG oslo_db.sqlalchemy.engines [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=53810) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 638.168188] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writer thread running {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 638.194261] nova-conductor[53810]: DEBUG nova.quota [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Getting quotas for project b8d93352efbf41be917afa7c0b5a1ed3. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 638.198410] nova-conductor[53810]: DEBUG nova.quota [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Getting quotas for user 92f701ef37364c629ed2fed539cda2cf and project b8d93352efbf41be917afa7c0b5a1ed3. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 638.210574] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] [instance: 224afc94-367d-4d30-95fb-b8b865f56eb9] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 638.211987] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 638.212332] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 638.212526] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 638.221685] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] [instance: 224afc94-367d-4d30-95fb-b8b865f56eb9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 638.222420] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 638.222647] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 638.222822] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 638.262783] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 638.262969] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 638.263153] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 638.263466] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Acquiring lock "compute-rpcapi-router" {{(pid=53810) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}} [ 638.263709] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Acquired lock "compute-rpcapi-router" {{(pid=53810) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}} [ 638.264162] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bb2b0298-9ce3-45bd-ac7c-4e04fda66793 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 638.264368] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bb2b0298-9ce3-45bd-ac7c-4e04fda66793 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 638.264694] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bb2b0298-9ce3-45bd-ac7c-4e04fda66793 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 638.264924] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bb2b0298-9ce3-45bd-ac7c-4e04fda66793 None None] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 638.265134] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bb2b0298-9ce3-45bd-ac7c-4e04fda66793 None None] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 638.265285] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bb2b0298-9ce3-45bd-ac7c-4e04fda66793 None None] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 638.272303] nova-conductor[53810]: INFO nova.compute.rpcapi [None req-bb2b0298-9ce3-45bd-ac7c-4e04fda66793 None None] Automatically selected compute RPC version 6.2 from minimum service version 66 [ 638.272778] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bb2b0298-9ce3-45bd-ac7c-4e04fda66793 None None] Releasing lock "compute-rpcapi-router" {{(pid=53810) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}} [ 638.916585] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Took 0.20 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 638.934275] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 638.936467] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 638.936467] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 639.012603] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 639.015040] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 639.015040] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 639.018067] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 639.019249] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 639.019249] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 639.034737] nova-conductor[53810]: DEBUG nova.quota [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Getting quotas for project 9bf93d58be7b48c298cc4ba7d350b918. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 639.040958] nova-conductor[53810]: DEBUG nova.quota [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Getting quotas for user 6ec8ca0476d2409cb087e4d3e92fdd7d and project 9bf93d58be7b48c298cc4ba7d350b918. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 639.050749] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] [instance: 0b4e2497-f306-4bfc-b862-26e53e827c16] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 639.050749] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 639.050749] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 639.050749] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 639.054677] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] [instance: 0b4e2497-f306-4bfc-b862-26e53e827c16] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 639.055380] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 639.056015] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 639.056015] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 639.087299] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 639.087569] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 639.087769] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 640.396874] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 640.413783] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 640.413985] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 640.414189] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 640.472584] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 640.472584] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 640.472584] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 640.472584] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 640.472882] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 640.472882] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 640.485462] nova-conductor[53810]: DEBUG nova.quota [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Getting quotas for project 870604e0902d43528453a2950d1080d3. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 640.491013] nova-conductor[53810]: DEBUG nova.quota [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Getting quotas for user bfae70c3a6a94df4a6689ac18626fe37 and project 870604e0902d43528453a2950d1080d3. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 640.497993] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] [instance: 10f688a0-63af-496f-93dd-794083acf94b] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 640.498730] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 640.499097] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 640.500696] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 640.507532] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] [instance: 10f688a0-63af-496f-93dd-794083acf94b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 640.507532] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 640.507532] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 640.507763] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 640.530307] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 640.533021] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 640.533021] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 643.270020] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 643.286231] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 643.286231] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 643.286231] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 643.337277] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 643.337786] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 643.338036] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 643.338882] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 643.338882] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 643.339035] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 643.356683] nova-conductor[53809]: DEBUG nova.quota [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Getting quotas for project 36ef9b8223f9482e822ab862327f5a4b. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 643.359424] nova-conductor[53809]: DEBUG nova.quota [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Getting quotas for user d18b73c168694833848a4b8625885086 and project 36ef9b8223f9482e822ab862327f5a4b. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 643.372227] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] [instance: 025daa70-33c7-4a0d-addf-94b680ad8c4e] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 643.373339] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 643.373492] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 643.373694] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 643.383984] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] [instance: 025daa70-33c7-4a0d-addf-94b680ad8c4e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 643.384921] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 643.385016] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 643.385203] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 643.410214] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 643.410431] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 643.410611] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 649.836746] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] [instance: 0b4e2497-f306-4bfc-b862-26e53e827c16] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 0066a37e-57c5-4cb7-a8b6-57709c4e4cc3, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 0b4e2497-f306-4bfc-b862-26e53e827c16 was re-scheduled: Binding failed for port 0066a37e-57c5-4cb7-a8b6-57709c4e4cc3, please check neutron logs for more information.\n'] [ 649.838980] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 649.840924] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0b4e2497-f306-4bfc-b862-26e53e827c16.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0b4e2497-f306-4bfc-b862-26e53e827c16. [ 649.844885] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] [instance: 0b4e2497-f306-4bfc-b862-26e53e827c16] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0b4e2497-f306-4bfc-b862-26e53e827c16. [ 649.967417] nova-conductor[53810]: DEBUG nova.network.neutron [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] [instance: 0b4e2497-f306-4bfc-b862-26e53e827c16] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 650.223246] nova-conductor[53810]: DEBUG nova.network.neutron [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] [instance: 0b4e2497-f306-4bfc-b862-26e53e827c16] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 650.234793] nova-conductor[53810]: DEBUG nova.network.neutron [None req-8febf802-3e3a-4c25-9d0a-bac645a436e3 tempest-ServerExternalEventsTest-723285954 tempest-ServerExternalEventsTest-723285954-project-member] [instance: 0b4e2497-f306-4bfc-b862-26e53e827c16] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 650.480357] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell0:SELECT=9 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 650.508700] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=30,nova_cell1:SAVEPOINT=3,nova_cell1:INSERT=60,nova_cell1:RELEASE=3 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 650.525428] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=35,nova_api:UPDATE=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 650.526929] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=36,nova_api:UPDATE=1,nova_api:DELETE=3 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 650.924224] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] [instance: 10f688a0-63af-496f-93dd-794083acf94b] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 54a2734c-5736-4c20-a208-d59449f937f8, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 10f688a0-63af-496f-93dd-794083acf94b was re-scheduled: Binding failed for port 54a2734c-5736-4c20-a208-d59449f937f8, please check neutron logs for more information.\n'] [ 650.924844] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 650.925088] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 10f688a0-63af-496f-93dd-794083acf94b.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 10f688a0-63af-496f-93dd-794083acf94b. [ 650.925416] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] [instance: 10f688a0-63af-496f-93dd-794083acf94b] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 10f688a0-63af-496f-93dd-794083acf94b. [ 650.954110] nova-conductor[53810]: DEBUG nova.network.neutron [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] [instance: 10f688a0-63af-496f-93dd-794083acf94b] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 651.005292] nova-conductor[53810]: DEBUG nova.network.neutron [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] [instance: 10f688a0-63af-496f-93dd-794083acf94b] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 651.012193] nova-conductor[53810]: DEBUG nova.network.neutron [None req-8d7088f1-7eb7-4dee-9946-47c07d4fd4c9 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] [instance: 10f688a0-63af-496f-93dd-794083acf94b] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 651.590158] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] [instance: 224afc94-367d-4d30-95fb-b8b865f56eb9] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 88a8eeb7-54a6-49a9-aae8-984e651885b1, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 224afc94-367d-4d30-95fb-b8b865f56eb9 was re-scheduled: Binding failed for port 88a8eeb7-54a6-49a9-aae8-984e651885b1, please check neutron logs for more information.\n'] [ 651.591660] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 651.591660] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 224afc94-367d-4d30-95fb-b8b865f56eb9.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 224afc94-367d-4d30-95fb-b8b865f56eb9. [ 651.592110] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] [instance: 224afc94-367d-4d30-95fb-b8b865f56eb9] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 224afc94-367d-4d30-95fb-b8b865f56eb9. [ 651.623132] nova-conductor[53810]: DEBUG nova.network.neutron [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] [instance: 224afc94-367d-4d30-95fb-b8b865f56eb9] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 651.690790] nova-conductor[53810]: DEBUG nova.network.neutron [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] [instance: 224afc94-367d-4d30-95fb-b8b865f56eb9] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 651.700360] nova-conductor[53810]: DEBUG nova.network.neutron [None req-3ba9f10e-7ab1-445e-994d-cf5cea16be51 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] [instance: 224afc94-367d-4d30-95fb-b8b865f56eb9] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 652.067121] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Took 0.21 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 652.092937] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 652.094395] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 652.094395] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 652.148257] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 652.148585] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 652.149033] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 652.149445] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 652.149634] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 652.150882] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 652.163210] nova-conductor[53810]: DEBUG nova.quota [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Getting quotas for project 26c233a104004a87a12bd1f2fd2c0dbb. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 652.165907] nova-conductor[53810]: DEBUG nova.quota [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Getting quotas for user 91295abcd6c74795abb5da3e18f72f38 and project 26c233a104004a87a12bd1f2fd2c0dbb. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 652.174945] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] [instance: 3d2438d5-9a7c-4cc6-b229-ab2389a78801] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 652.174945] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 652.174945] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 652.174945] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 652.180296] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] [instance: 3d2438d5-9a7c-4cc6-b229-ab2389a78801] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 652.181610] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 652.181610] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 652.181610] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 652.216112] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 652.219497] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 652.219497] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 653.345831] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell0:SELECT=7 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 653.388849] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=24,nova_cell1:SAVEPOINT=2,nova_cell1:INSERT=41,nova_cell1:RELEASE=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 653.398369] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=23,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 653.402035] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=24,nova_api:DELETE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 655.704823] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] [instance: 025daa70-33c7-4a0d-addf-94b680ad8c4e] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 3aa58b3c-4708-4d52-ac7e-6e0f2eee48f5, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 025daa70-33c7-4a0d-addf-94b680ad8c4e was re-scheduled: Binding failed for port 3aa58b3c-4708-4d52-ac7e-6e0f2eee48f5, please check neutron logs for more information.\n'] [ 655.706081] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 655.706411] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 025daa70-33c7-4a0d-addf-94b680ad8c4e.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 025daa70-33c7-4a0d-addf-94b680ad8c4e. [ 655.706658] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] [instance: 025daa70-33c7-4a0d-addf-94b680ad8c4e] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 025daa70-33c7-4a0d-addf-94b680ad8c4e. [ 655.733434] nova-conductor[53810]: DEBUG nova.network.neutron [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] [instance: 025daa70-33c7-4a0d-addf-94b680ad8c4e] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 655.779648] nova-conductor[53810]: DEBUG nova.network.neutron [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] [instance: 025daa70-33c7-4a0d-addf-94b680ad8c4e] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 655.785842] nova-conductor[53810]: DEBUG nova.network.neutron [None req-3f1b0dd4-4f52-4264-9da2-e4bbd58a9771 tempest-ImagesOneServerNegativeTestJSON-1421586542 tempest-ImagesOneServerNegativeTestJSON-1421586542-project-member] [instance: 025daa70-33c7-4a0d-addf-94b680ad8c4e] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 657.022856] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 657.043256] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 657.043256] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 657.043256] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 657.094106] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 657.094212] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 657.094975] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 657.094975] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 657.094975] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 657.095524] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 657.112396] nova-conductor[53810]: DEBUG nova.quota [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Getting quotas for project c85d469fb8e045f7b6981676c526d780. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 657.115564] nova-conductor[53810]: DEBUG nova.quota [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Getting quotas for user 2d0376ef3212459b883f3b757a17316f and project c85d469fb8e045f7b6981676c526d780. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 657.121529] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] [instance: c3fd73f2-027d-42d3-ad53-4925403a9c92] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 657.121529] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 657.121802] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 657.122134] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 657.128904] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] [instance: c3fd73f2-027d-42d3-ad53-4925403a9c92] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 657.129628] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 657.129851] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 657.130032] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 657.160922] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 657.160922] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 657.160922] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 658.849183] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Took 0.24 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 658.871822] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 658.871822] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 658.872010] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 658.919123] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 658.919903] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 658.919903] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 658.920072] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 658.920274] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 658.920515] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 658.939086] nova-conductor[53809]: DEBUG nova.quota [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Getting quotas for project 00123cdebdcf4b01ad920d353fcacff4. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 658.942977] nova-conductor[53809]: DEBUG nova.quota [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Getting quotas for user 564a426ea7b54c38a2bb07a08fc935be and project 00123cdebdcf4b01ad920d353fcacff4. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 658.956660] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: fb8aee53-11e7-4e32-9225-06671bb511d7] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 658.956660] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 658.956660] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 658.956660] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 658.970146] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: fb8aee53-11e7-4e32-9225-06671bb511d7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 658.970146] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 658.970146] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 658.970146] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 658.992272] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 658.992531] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 658.992716] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 659.428733] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] [instance: 3d2438d5-9a7c-4cc6-b229-ab2389a78801] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port e2841af4-5a95-4a05-bbd9-6e514778a012, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 3d2438d5-9a7c-4cc6-b229-ab2389a78801 was re-scheduled: Binding failed for port e2841af4-5a95-4a05-bbd9-6e514778a012, please check neutron logs for more information.\n'] [ 659.433940] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 659.433940] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3d2438d5-9a7c-4cc6-b229-ab2389a78801.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3d2438d5-9a7c-4cc6-b229-ab2389a78801. [ 659.433940] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] [instance: 3d2438d5-9a7c-4cc6-b229-ab2389a78801] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3d2438d5-9a7c-4cc6-b229-ab2389a78801. [ 659.521021] nova-conductor[53809]: DEBUG nova.network.neutron [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] [instance: 3d2438d5-9a7c-4cc6-b229-ab2389a78801] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 659.562805] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Took 0.20 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 659.579513] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 659.579740] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 659.579912] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 659.637785] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 659.638979] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 659.638979] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 659.638979] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 659.639217] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 659.639217] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 659.657629] nova-conductor[53810]: DEBUG nova.quota [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Getting quotas for project 5bf81a56b9294a7a846c62485175a74c. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 659.661555] nova-conductor[53810]: DEBUG nova.quota [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Getting quotas for user ff121f6262f3409e93a2b29db03b6326 and project 5bf81a56b9294a7a846c62485175a74c. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 659.669279] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] [instance: 4bce210a-e6bc-4c67-ad48-397422291467] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 659.670380] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 659.670380] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 659.670380] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 659.677450] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] [instance: 4bce210a-e6bc-4c67-ad48-397422291467] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 659.677450] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 659.677450] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 659.677450] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 659.712327] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 659.712712] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 659.712870] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 659.858257] nova-conductor[53809]: DEBUG nova.network.neutron [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] [instance: 3d2438d5-9a7c-4cc6-b229-ab2389a78801] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 659.877307] nova-conductor[53809]: DEBUG nova.network.neutron [None req-ef9b7a46-0c2e-40a2-ab82-a8e14bb3ba53 tempest-ServersTestFqdnHostnames-151657379 tempest-ServersTestFqdnHostnames-151657379-project-member] [instance: 3d2438d5-9a7c-4cc6-b229-ab2389a78801] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 660.572510] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Took 0.21 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 660.593305] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 660.593793] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 660.594175] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 660.668765] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 660.668998] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 660.669198] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 660.670074] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 660.670074] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 660.670074] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 660.681409] nova-conductor[53810]: DEBUG nova.quota [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Getting quotas for project c39211357c76487183cc1181d114332a. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 660.684151] nova-conductor[53810]: DEBUG nova.quota [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Getting quotas for user 1d6f330f0a944ff09915d62b9fe32137 and project c39211357c76487183cc1181d114332a. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 660.690043] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] [instance: 7c682f87-0bf4-460c-97a7-bd369d91a6a2] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 660.690626] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 660.690806] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 660.693760] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 660.698652] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] [instance: 7c682f87-0bf4-460c-97a7-bd369d91a6a2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 660.699458] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 660.699500] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 660.699709] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 660.721828] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 660.722163] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 660.722287] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 662.341837] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Took 0.30 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 662.379697] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 662.380048] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 662.380681] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 662.448092] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 662.448353] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 662.448530] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 662.449047] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 662.449161] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 662.449337] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 662.467477] nova-conductor[53809]: DEBUG nova.quota [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Getting quotas for project 3e1db803f0ff4f29bb70e0a0d94c57e0. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 662.470708] nova-conductor[53809]: DEBUG nova.quota [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Getting quotas for user e64ca57e567146098521cd7356b9e3e2 and project 3e1db803f0ff4f29bb70e0a0d94c57e0. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 662.477362] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: edb47400-6749-4235-bbb1-ffa648f3dba5] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 662.478792] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 662.478792] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 662.478952] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 662.489253] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: edb47400-6749-4235-bbb1-ffa648f3dba5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 662.489918] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 662.490221] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 662.490484] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 662.521718] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 662.522052] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 662.522733] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 663.630904] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Took 0.24 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 663.645589] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 663.645881] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 663.646080] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 663.680934] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 663.681314] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 663.681561] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 663.682049] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 663.682373] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 663.682610] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 663.696316] nova-conductor[53810]: DEBUG nova.quota [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Getting quotas for project 00123cdebdcf4b01ad920d353fcacff4. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 663.699031] nova-conductor[53810]: DEBUG nova.quota [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Getting quotas for user 564a426ea7b54c38a2bb07a08fc935be and project 00123cdebdcf4b01ad920d353fcacff4. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 663.706076] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: 05067c00-8595-412c-ad68-095cba5bf6da] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 663.706686] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 663.706969] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 663.707233] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 663.711570] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: 05067c00-8595-412c-ad68-095cba5bf6da] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 663.712268] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 663.712497] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 663.712659] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 663.734933] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 663.735200] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 663.735376] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 664.081868] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Took 0.15 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 664.106311] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 664.106311] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 664.106311] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 664.177225] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 664.177494] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 664.177674] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 664.178119] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 664.178330] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 664.178510] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 664.202558] nova-conductor[53809]: DEBUG nova.quota [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Getting quotas for project b8d93352efbf41be917afa7c0b5a1ed3. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 664.206589] nova-conductor[53809]: DEBUG nova.quota [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Getting quotas for user 92f701ef37364c629ed2fed539cda2cf and project b8d93352efbf41be917afa7c0b5a1ed3. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 664.222123] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] [instance: 605eecc4-0fff-44bc-9d79-799c4762707a] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 664.222123] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 664.222123] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 664.222123] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 664.227946] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] [instance: 605eecc4-0fff-44bc-9d79-799c4762707a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 664.228097] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 664.228313] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 664.228485] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 664.273924] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 664.274455] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 664.274703] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 666.032311] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 666.050936] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 666.051501] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 666.051699] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 666.095331] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 666.095588] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 666.095771] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 666.096165] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 666.096360] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 666.096532] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 666.106955] nova-conductor[53810]: DEBUG nova.quota [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Getting quotas for project 177860dab437443984a62a6110755d3b. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 666.109127] nova-conductor[53810]: DEBUG nova.quota [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Getting quotas for user e462b9a9c54f42efb88b614e2b31eda3 and project 177860dab437443984a62a6110755d3b. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 666.120556] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] [instance: e32ca59c-5bce-40e5-85e9-0979e673688a] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 666.121219] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 666.121321] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 666.121499] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 666.129615] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] [instance: e32ca59c-5bce-40e5-85e9-0979e673688a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 666.130417] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 666.130636] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 666.130813] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 666.147178] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 666.147427] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 666.147607] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 666.703622] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] [instance: c3fd73f2-027d-42d3-ad53-4925403a9c92] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 6787e179-849f-4357-9cfb-6b6f599dc8c9, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance c3fd73f2-027d-42d3-ad53-4925403a9c92 was re-scheduled: Binding failed for port 6787e179-849f-4357-9cfb-6b6f599dc8c9, please check neutron logs for more information.\n'] [ 666.704481] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 666.704740] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c3fd73f2-027d-42d3-ad53-4925403a9c92.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c3fd73f2-027d-42d3-ad53-4925403a9c92. [ 666.705097] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] [instance: c3fd73f2-027d-42d3-ad53-4925403a9c92] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c3fd73f2-027d-42d3-ad53-4925403a9c92. [ 666.752495] nova-conductor[53809]: DEBUG nova.network.neutron [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] [instance: c3fd73f2-027d-42d3-ad53-4925403a9c92] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 666.816734] nova-conductor[53809]: DEBUG nova.network.neutron [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] [instance: c3fd73f2-027d-42d3-ad53-4925403a9c92] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 666.821741] nova-conductor[53809]: DEBUG nova.network.neutron [None req-55fb50b8-fa11-4d57-81fc-29c0dd1f637e tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] [instance: c3fd73f2-027d-42d3-ad53-4925403a9c92] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 669.236970] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: fb8aee53-11e7-4e32-9225-06671bb511d7] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 4f72723f-9f91-4066-a9a6-fe0bf74afafd, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance fb8aee53-11e7-4e32-9225-06671bb511d7 was re-scheduled: Binding failed for port 4f72723f-9f91-4066-a9a6-fe0bf74afafd, please check neutron logs for more information.\n'] [ 669.237580] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 669.237844] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fb8aee53-11e7-4e32-9225-06671bb511d7.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fb8aee53-11e7-4e32-9225-06671bb511d7. [ 669.238169] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: fb8aee53-11e7-4e32-9225-06671bb511d7] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fb8aee53-11e7-4e32-9225-06671bb511d7. [ 669.261878] nova-conductor[53809]: DEBUG nova.network.neutron [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: fb8aee53-11e7-4e32-9225-06671bb511d7] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 669.325198] nova-conductor[53809]: DEBUG nova.network.neutron [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: fb8aee53-11e7-4e32-9225-06671bb511d7] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 669.336757] nova-conductor[53809]: DEBUG nova.network.neutron [None req-4b07bd65-a3d3-4a63-a463-ac2e8be1c6b1 tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: fb8aee53-11e7-4e32-9225-06671bb511d7] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 670.556585] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Took 0.25 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 670.577175] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 670.577434] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 670.577610] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 670.652400] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 670.652655] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 670.652833] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 670.654988] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 670.655184] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 670.655359] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 670.680079] nova-conductor[53809]: DEBUG nova.quota [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Getting quotas for project 00123cdebdcf4b01ad920d353fcacff4. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 670.683521] nova-conductor[53809]: DEBUG nova.quota [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Getting quotas for user 564a426ea7b54c38a2bb07a08fc935be and project 00123cdebdcf4b01ad920d353fcacff4. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 670.690721] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: daf5a018-8f3c-4aa3-bff6-d185c9f125b3] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 670.692803] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 670.693149] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 670.693484] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 670.706019] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: daf5a018-8f3c-4aa3-bff6-d185c9f125b3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 670.706019] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 670.706019] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 670.706019] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 670.724690] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 670.725141] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 670.725366] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 671.025295] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] [instance: 7c682f87-0bf4-460c-97a7-bd369d91a6a2] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 9b109a6b-fa18-4dfc-9ac9-4beedf6be738, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 7c682f87-0bf4-460c-97a7-bd369d91a6a2 was re-scheduled: Binding failed for port 9b109a6b-fa18-4dfc-9ac9-4beedf6be738, please check neutron logs for more information.\n'] [ 671.025295] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 671.025295] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7c682f87-0bf4-460c-97a7-bd369d91a6a2.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7c682f87-0bf4-460c-97a7-bd369d91a6a2. [ 671.025560] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] [instance: 7c682f87-0bf4-460c-97a7-bd369d91a6a2] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7c682f87-0bf4-460c-97a7-bd369d91a6a2. [ 671.052726] nova-conductor[53809]: DEBUG nova.network.neutron [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] [instance: 7c682f87-0bf4-460c-97a7-bd369d91a6a2] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 671.123549] nova-conductor[53809]: DEBUG nova.network.neutron [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] [instance: 7c682f87-0bf4-460c-97a7-bd369d91a6a2] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 671.132677] nova-conductor[53809]: DEBUG nova.network.neutron [None req-94cae6c2-bef0-4f51-b7c0-29c244366628 tempest-ServerMetadataTestJSON-618169477 tempest-ServerMetadataTestJSON-618169477-project-member] [instance: 7c682f87-0bf4-460c-97a7-bd369d91a6a2] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 673.091301] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] [instance: 4bce210a-e6bc-4c67-ad48-397422291467] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port dc8ac46f-f616-4f92-ad64-f212950b7634, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 4bce210a-e6bc-4c67-ad48-397422291467 was re-scheduled: Binding failed for port dc8ac46f-f616-4f92-ad64-f212950b7634, please check neutron logs for more information.\n'] [ 673.092770] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 673.093163] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4bce210a-e6bc-4c67-ad48-397422291467.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4bce210a-e6bc-4c67-ad48-397422291467. [ 673.093577] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] [instance: 4bce210a-e6bc-4c67-ad48-397422291467] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4bce210a-e6bc-4c67-ad48-397422291467. [ 673.125452] nova-conductor[53809]: DEBUG nova.network.neutron [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] [instance: 4bce210a-e6bc-4c67-ad48-397422291467] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 673.162787] nova-conductor[53809]: DEBUG nova.network.neutron [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] [instance: 4bce210a-e6bc-4c67-ad48-397422291467] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 673.167420] nova-conductor[53809]: DEBUG nova.network.neutron [None req-f2c19f30-b06b-494a-855f-84876c33f6d0 tempest-ServersAdminNegativeTestJSON-266339737 tempest-ServersAdminNegativeTestJSON-266339737-project-member] [instance: 4bce210a-e6bc-4c67-ad48-397422291467] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 673.753392] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: 05067c00-8595-412c-ad68-095cba5bf6da] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b4c6a589-d627-4d9e-b784-1847a9fab568, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 05067c00-8595-412c-ad68-095cba5bf6da was re-scheduled: Binding failed for port b4c6a589-d627-4d9e-b784-1847a9fab568, please check neutron logs for more information.\n'] [ 673.755827] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 673.755827] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 05067c00-8595-412c-ad68-095cba5bf6da.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 05067c00-8595-412c-ad68-095cba5bf6da. [ 673.755827] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: 05067c00-8595-412c-ad68-095cba5bf6da] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 05067c00-8595-412c-ad68-095cba5bf6da. [ 673.813442] nova-conductor[53809]: DEBUG nova.network.neutron [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: 05067c00-8595-412c-ad68-095cba5bf6da] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 673.864960] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] [instance: 605eecc4-0fff-44bc-9d79-799c4762707a] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 8a72056c-1a3f-41ec-8540-27f0817eba1c, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 605eecc4-0fff-44bc-9d79-799c4762707a was re-scheduled: Binding failed for port 8a72056c-1a3f-41ec-8540-27f0817eba1c, please check neutron logs for more information.\n'] [ 673.865720] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 673.868127] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 605eecc4-0fff-44bc-9d79-799c4762707a.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 605eecc4-0fff-44bc-9d79-799c4762707a. [ 673.868127] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] [instance: 605eecc4-0fff-44bc-9d79-799c4762707a] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 605eecc4-0fff-44bc-9d79-799c4762707a. [ 673.895700] nova-conductor[53810]: DEBUG nova.network.neutron [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] [instance: 605eecc4-0fff-44bc-9d79-799c4762707a] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 673.901099] nova-conductor[53809]: DEBUG nova.network.neutron [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: 05067c00-8595-412c-ad68-095cba5bf6da] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 673.909564] nova-conductor[53809]: DEBUG nova.network.neutron [None req-6a5403d9-affe-45b5-a051-50d6829a768a tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: 05067c00-8595-412c-ad68-095cba5bf6da] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 674.154126] nova-conductor[53810]: DEBUG nova.network.neutron [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] [instance: 605eecc4-0fff-44bc-9d79-799c4762707a] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 674.162545] nova-conductor[53810]: DEBUG nova.network.neutron [None req-77e04db6-3318-4c6e-a19b-9e54ea667fc7 tempest-DeleteServersAdminTestJSON-2134891279 tempest-DeleteServersAdminTestJSON-2134891279-project-member] [instance: 605eecc4-0fff-44bc-9d79-799c4762707a] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 675.236588] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Took 0.18 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 675.272710] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 675.272952] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 675.273162] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 675.336581] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 675.336581] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 675.336766] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 675.337067] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 675.337512] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 675.337512] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 675.353661] nova-conductor[53809]: DEBUG nova.quota [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Getting quotas for project 3511963fac0e4f2db836bdd43d779946. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 675.356552] nova-conductor[53809]: DEBUG nova.quota [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Getting quotas for user 15f09ca9b4fc4a6f9100a73dac11dcc2 and project 3511963fac0e4f2db836bdd43d779946. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 675.366097] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] [instance: 4998340d-3afc-4fc7-bc2a-57913b534d36] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 675.366279] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 675.366506] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 675.367725] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 675.377486] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] [instance: 4998340d-3afc-4fc7-bc2a-57913b534d36] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 675.378326] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 675.379377] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 675.379377] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 675.405745] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 675.406159] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 675.406366] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 676.044548] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Took 0.21 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 676.077755] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 676.077755] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 676.077969] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 676.101741] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell0:SELECT=12 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 676.135171] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=54,nova_cell1:SAVEPOINT=5,nova_cell1:INSERT=119,nova_cell1:RELEASE=5 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 676.141285] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=72,nova_api:UPDATE=5,nova_api:DELETE=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 676.148840] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 676.148948] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=72,nova_api:DELETE=4,nova_api:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 676.148984] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 676.148984] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 676.149354] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 676.149546] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 676.149831] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 676.163387] nova-conductor[53809]: DEBUG nova.quota [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Getting quotas for project 24fe9a8f68c746dea82c0ebcd5408e85. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 676.166190] nova-conductor[53809]: DEBUG nova.quota [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Getting quotas for user a1b212297e564783987dc59263fc50c8 and project 24fe9a8f68c746dea82c0ebcd5408e85. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 676.174844] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] [instance: f66615e9-2e8c-43d9-be73-154398e26934] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 676.175495] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 676.175710] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 676.175886] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 676.181890] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] [instance: f66615e9-2e8c-43d9-be73-154398e26934] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 676.182810] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 676.182917] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 676.183037] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 676.216025] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: edb47400-6749-4235-bbb1-ffa648f3dba5] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 2368fddb-20b2-40b3-b7d3-7d1fce015c95, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance edb47400-6749-4235-bbb1-ffa648f3dba5 was re-scheduled: Binding failed for port 2368fddb-20b2-40b3-b7d3-7d1fce015c95, please check neutron logs for more information.\n'] [ 676.216371] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 676.216600] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance edb47400-6749-4235-bbb1-ffa648f3dba5.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance edb47400-6749-4235-bbb1-ffa648f3dba5. [ 676.217225] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: edb47400-6749-4235-bbb1-ffa648f3dba5] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance edb47400-6749-4235-bbb1-ffa648f3dba5. [ 676.237567] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 676.237567] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 676.237567] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 676.260804] nova-conductor[53809]: DEBUG nova.network.neutron [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: edb47400-6749-4235-bbb1-ffa648f3dba5] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 676.333917] nova-conductor[53809]: DEBUG nova.network.neutron [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: edb47400-6749-4235-bbb1-ffa648f3dba5] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 676.354444] nova-conductor[53809]: DEBUG nova.network.neutron [None req-9006b775-fce8-4ed2-a697-d3002b236004 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: edb47400-6749-4235-bbb1-ffa648f3dba5] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 678.544275] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Took 0.18 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 678.563300] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 678.563537] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 678.563720] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 678.608252] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 678.608549] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 678.608746] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 678.609181] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 678.609395] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 678.609581] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 678.619122] nova-conductor[53810]: DEBUG nova.quota [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Getting quotas for project 942a20ed43a149499564de0ae5103aa5. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 678.622085] nova-conductor[53810]: DEBUG nova.quota [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Getting quotas for user 37ebdb44c9774c7cb2451adfa198e770 and project 942a20ed43a149499564de0ae5103aa5. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 678.627961] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] [instance: 429ae3c8-1646-413e-a821-0d37a2ec20e4] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 678.628419] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 678.628626] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 678.628801] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 678.632905] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] [instance: 429ae3c8-1646-413e-a821-0d37a2ec20e4] block_device_mapping [BlockDeviceMapping(attachment_id=850cb627-f806-4c2c-b62d-33905c49bec2,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='75a1df95-7638-4f43-a717-35b7645995ea',volume_size=1,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 678.633592] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 678.633799] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 678.633993] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 678.652894] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 678.653142] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 678.653370] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 679.112668] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] [instance: e32ca59c-5bce-40e5-85e9-0979e673688a] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 8a86f436-b6f2-45a2-872f-ea805c774367, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance e32ca59c-5bce-40e5-85e9-0979e673688a was re-scheduled: Binding failed for port 8a86f436-b6f2-45a2-872f-ea805c774367, please check neutron logs for more information.\n'] [ 679.113289] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 679.113523] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e32ca59c-5bce-40e5-85e9-0979e673688a.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e32ca59c-5bce-40e5-85e9-0979e673688a. [ 679.113737] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] [instance: e32ca59c-5bce-40e5-85e9-0979e673688a] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e32ca59c-5bce-40e5-85e9-0979e673688a. [ 679.152982] nova-conductor[53809]: DEBUG nova.network.neutron [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] [instance: e32ca59c-5bce-40e5-85e9-0979e673688a] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 679.264316] nova-conductor[53809]: DEBUG nova.network.neutron [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] [instance: e32ca59c-5bce-40e5-85e9-0979e673688a] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 679.270114] nova-conductor[53809]: DEBUG nova.network.neutron [None req-468bca86-a3ef-4b10-89e0-41b51510fb04 tempest-TenantUsagesTestJSON-325677525 tempest-TenantUsagesTestJSON-325677525-project-member] [instance: e32ca59c-5bce-40e5-85e9-0979e673688a] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 680.472761] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: daf5a018-8f3c-4aa3-bff6-d185c9f125b3] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port bae68af0-39c4-46fb-80c7-7a45da13e70a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance daf5a018-8f3c-4aa3-bff6-d185c9f125b3 was re-scheduled: Binding failed for port bae68af0-39c4-46fb-80c7-7a45da13e70a, please check neutron logs for more information.\n'] [ 680.472761] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 680.472761] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance daf5a018-8f3c-4aa3-bff6-d185c9f125b3.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance daf5a018-8f3c-4aa3-bff6-d185c9f125b3. [ 680.474581] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: daf5a018-8f3c-4aa3-bff6-d185c9f125b3] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance daf5a018-8f3c-4aa3-bff6-d185c9f125b3. [ 680.520526] nova-conductor[53810]: DEBUG nova.network.neutron [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: daf5a018-8f3c-4aa3-bff6-d185c9f125b3] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 680.695438] nova-conductor[53810]: DEBUG nova.network.neutron [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: daf5a018-8f3c-4aa3-bff6-d185c9f125b3] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 680.701018] nova-conductor[53810]: DEBUG nova.network.neutron [None req-0fdc3667-2428-4992-9035-a94f07f007eb tempest-ListServerFiltersTestJSON-1003122498 tempest-ListServerFiltersTestJSON-1003122498-project-member] [instance: daf5a018-8f3c-4aa3-bff6-d185c9f125b3] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 681.031210] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 681.046552] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 681.047026] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 681.047447] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 681.100821] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 681.103120] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 681.103120] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 681.103120] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 681.103120] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 681.103120] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 681.120975] nova-conductor[53810]: DEBUG nova.quota [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Getting quotas for project 870604e0902d43528453a2950d1080d3. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 681.123588] nova-conductor[53810]: DEBUG nova.quota [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Getting quotas for user bfae70c3a6a94df4a6689ac18626fe37 and project 870604e0902d43528453a2950d1080d3. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 681.130604] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] [instance: 0e994334-34bc-4c80-8825-8cfec714f81f] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 681.130604] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 681.130604] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 681.130604] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 681.138446] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] [instance: 0e994334-34bc-4c80-8825-8cfec714f81f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 681.138446] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 681.138446] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 681.138446] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 681.161345] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 681.161345] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 681.161345] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 685.088329] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 685.105330] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 685.105330] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 685.105636] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 685.154202] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 685.154456] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 685.154660] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 685.155073] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 685.155301] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 685.155474] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 685.166321] nova-conductor[53809]: DEBUG nova.quota [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Getting quotas for project 478c83960c4a4bf1b9e93a534ea7528e. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 685.169748] nova-conductor[53809]: DEBUG nova.quota [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Getting quotas for user d6a1d54790264fab9950cf8fc84a0786 and project 478c83960c4a4bf1b9e93a534ea7528e. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 685.176917] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] [instance: a41cb33f-8340-4b15-b19d-0a7b9396eae7] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 685.177785] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 685.178268] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 685.178268] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 685.182773] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] [instance: a41cb33f-8340-4b15-b19d-0a7b9396eae7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 685.183512] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 685.183850] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 685.184240] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 685.222029] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 685.222029] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 685.222029] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 685.240495] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=781,nova_cell1:UPDATE=226,nova_cell1:INSERT=26,nova_cell1:SAVEPOINT=34,nova_cell1:RELEASE=34 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 686.194161] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 686.208413] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 686.209324] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 686.209623] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 686.272889] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 686.273196] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 686.275331] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 686.275331] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 686.275331] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 686.275633] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 686.289878] nova-conductor[53809]: DEBUG nova.quota [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Getting quotas for project fc06e5943835421bb7461a395d058b90. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 686.293359] nova-conductor[53809]: DEBUG nova.quota [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Getting quotas for user 4cedace28efb449abe5bc60d4f6b3b7c and project fc06e5943835421bb7461a395d058b90. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 686.299737] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] [instance: 8f553865-312c-4b6a-9383-43f2cbcd0b5e] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 686.300345] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 686.300662] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 686.300880] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 686.306646] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] [instance: 8f553865-312c-4b6a-9383-43f2cbcd0b5e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 686.307870] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 686.308175] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 686.308375] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 686.330496] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 686.330776] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 686.331030] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 688.289934] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=859,nova_cell1:UPDATE=238,nova_cell1:INSERT=40,nova_cell1:SAVEPOINT=37,nova_cell1:RELEASE=37 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 688.556968] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] [instance: f66615e9-2e8c-43d9-be73-154398e26934] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port cbbf3dd4-54e9-4d63-b499-569d917f7cbf, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance f66615e9-2e8c-43d9-be73-154398e26934 was re-scheduled: Binding failed for port cbbf3dd4-54e9-4d63-b499-569d917f7cbf, please check neutron logs for more information.\n'] [ 688.561032] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 688.561032] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f66615e9-2e8c-43d9-be73-154398e26934.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f66615e9-2e8c-43d9-be73-154398e26934. [ 688.561032] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] [instance: f66615e9-2e8c-43d9-be73-154398e26934] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f66615e9-2e8c-43d9-be73-154398e26934. [ 688.589690] nova-conductor[53810]: DEBUG nova.network.neutron [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] [instance: f66615e9-2e8c-43d9-be73-154398e26934] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 688.699665] nova-conductor[53810]: DEBUG nova.network.neutron [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] [instance: f66615e9-2e8c-43d9-be73-154398e26934] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 688.709075] nova-conductor[53810]: DEBUG nova.network.neutron [None req-f72ed652-bc39-4027-8d10-f95153c80ab0 tempest-VolumesAssistedSnapshotsTest-593068978 tempest-VolumesAssistedSnapshotsTest-593068978-project-member] [instance: f66615e9-2e8c-43d9-be73-154398e26934] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 689.653464] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] [instance: 4998340d-3afc-4fc7-bc2a-57913b534d36] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b242e1a9-96ac-4229-abd4-34b88a2ae9b0, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 4998340d-3afc-4fc7-bc2a-57913b534d36 was re-scheduled: Binding failed for port b242e1a9-96ac-4229-abd4-34b88a2ae9b0, please check neutron logs for more information.\n'] [ 689.654076] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 689.654348] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4998340d-3afc-4fc7-bc2a-57913b534d36.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4998340d-3afc-4fc7-bc2a-57913b534d36. [ 689.654744] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] [instance: 4998340d-3afc-4fc7-bc2a-57913b534d36] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4998340d-3afc-4fc7-bc2a-57913b534d36. [ 689.683654] nova-conductor[53809]: DEBUG nova.network.neutron [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] [instance: 4998340d-3afc-4fc7-bc2a-57913b534d36] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 689.771020] nova-conductor[53809]: DEBUG nova.network.neutron [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] [instance: 4998340d-3afc-4fc7-bc2a-57913b534d36] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 689.777412] nova-conductor[53809]: DEBUG nova.network.neutron [None req-93424e1c-6008-4427-8041-e87d2479003c tempest-ImagesOneServerTestJSON-2133586259 tempest-ImagesOneServerTestJSON-2133586259-project-member] [instance: 4998340d-3afc-4fc7-bc2a-57913b534d36] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 691.111059] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell0:SELECT=4 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 691.135545] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] [instance: 2afc8edb-3331-476a-bda3-4f8071461084] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 2afc8edb-3331-476a-bda3-4f8071461084 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 691.138728] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 691.138728] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2afc8edb-3331-476a-bda3-4f8071461084.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2afc8edb-3331-476a-bda3-4f8071461084. [ 691.138728] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-988e22c9-c6db-454a-ba6a-e2e8a94cdef7 tempest-ServersAdmin275Test-1748718107 tempest-ServersAdmin275Test-1748718107-project-member] [instance: 2afc8edb-3331-476a-bda3-4f8071461084] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2afc8edb-3331-476a-bda3-4f8071461084. [ 691.140878] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=18,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=37,nova_cell1:RELEASE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 691.152373] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=19 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 691.154874] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=29,nova_api:UPDATE=2,nova_api:DELETE=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 693.253356] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] [instance: 429ae3c8-1646-413e-a821-0d37a2ec20e4] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 0d60095c-bda7-480f-944a-27d6f3cfd665, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 429ae3c8-1646-413e-a821-0d37a2ec20e4 was re-scheduled: Binding failed for port 0d60095c-bda7-480f-944a-27d6f3cfd665, please check neutron logs for more information.\n'] [ 693.257801] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 693.257801] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 429ae3c8-1646-413e-a821-0d37a2ec20e4.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 429ae3c8-1646-413e-a821-0d37a2ec20e4. [ 693.258912] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] [instance: 429ae3c8-1646-413e-a821-0d37a2ec20e4] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 429ae3c8-1646-413e-a821-0d37a2ec20e4. [ 693.296971] nova-conductor[53809]: DEBUG nova.network.neutron [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] [instance: 429ae3c8-1646-413e-a821-0d37a2ec20e4] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 693.345425] nova-conductor[53809]: DEBUG nova.network.neutron [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] [instance: 429ae3c8-1646-413e-a821-0d37a2ec20e4] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 693.351996] nova-conductor[53809]: DEBUG nova.network.neutron [None req-5e8ee6c2-d77e-4d6a-8545-c4900ef915b6 tempest-ServerActionsV293TestJSON-231157793 tempest-ServerActionsV293TestJSON-231157793-project-member] [instance: 429ae3c8-1646-413e-a821-0d37a2ec20e4] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 693.634930] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Took 0.32 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 693.659612] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 693.660948] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 693.661622] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 693.724367] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 693.724609] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 693.724776] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 693.725159] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 693.725352] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 693.725521] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 693.736046] nova-conductor[53809]: DEBUG nova.quota [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Getting quotas for project 0ea0ee0c77d94ffe9ad2da96628a9075. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 693.739880] nova-conductor[53809]: DEBUG nova.quota [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Getting quotas for user 0afa592ffb1c4a7cb7389fa1d5b0e0e4 and project 0ea0ee0c77d94ffe9ad2da96628a9075. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 693.750133] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] [instance: 4a2752b8-156a-4ec2-810c-3d6e7bc8554e] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 693.750636] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 693.751052] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 693.751358] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 693.757315] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] [instance: 4a2752b8-156a-4ec2-810c-3d6e7bc8554e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 693.758062] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 693.758692] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 693.758692] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 693.777027] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 693.777319] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 693.777504] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 694.463217] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] [instance: 0e994334-34bc-4c80-8825-8cfec714f81f] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 3e6a70c5-5d66-4536-bff8-68f7bc1921c8, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 0e994334-34bc-4c80-8825-8cfec714f81f was re-scheduled: Binding failed for port 3e6a70c5-5d66-4536-bff8-68f7bc1921c8, please check neutron logs for more information.\n'] [ 694.466420] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 694.466420] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0e994334-34bc-4c80-8825-8cfec714f81f.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0e994334-34bc-4c80-8825-8cfec714f81f. [ 694.466420] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] [instance: 0e994334-34bc-4c80-8825-8cfec714f81f] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0e994334-34bc-4c80-8825-8cfec714f81f. [ 694.495884] nova-conductor[53809]: DEBUG nova.network.neutron [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] [instance: 0e994334-34bc-4c80-8825-8cfec714f81f] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 694.557216] nova-conductor[53809]: DEBUG nova.network.neutron [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] [instance: 0e994334-34bc-4c80-8825-8cfec714f81f] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 694.562417] nova-conductor[53809]: DEBUG nova.network.neutron [None req-c32066ce-073f-4505-a972-1a626174f3c3 tempest-MigrationsAdminTest-1447096877 tempest-MigrationsAdminTest-1447096877-project-member] [instance: 0e994334-34bc-4c80-8825-8cfec714f81f] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 695.687602] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Took 0.23 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 695.711096] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 695.711341] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 695.711513] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 695.779955] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 695.780223] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 695.780450] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 695.780871] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 695.781045] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 695.781329] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 695.804812] nova-conductor[53809]: DEBUG nova.quota [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Getting quotas for project 31e1deda33684cd08f46b5bf9e64b772. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 695.809663] nova-conductor[53809]: DEBUG nova.quota [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Getting quotas for user a8d0a84c83694e7d820bf8fa60bcef3f and project 31e1deda33684cd08f46b5bf9e64b772. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 695.817151] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] [instance: ad05c801-aff7-4a36-88e9-4994adfa3a8c] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 695.817643] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 695.817892] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 695.818137] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 695.827075] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] [instance: ad05c801-aff7-4a36-88e9-4994adfa3a8c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 695.827318] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 695.827540] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 695.831021] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 695.860911] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 695.861043] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 695.862187] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 697.222693] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 697.242438] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 697.242648] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 697.242832] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 697.349257] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 697.350053] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 697.350299] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 697.350709] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 697.350880] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 697.351413] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 697.364291] nova-conductor[53809]: DEBUG nova.quota [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Getting quotas for project 362a29354f714f0a8c002c5d933262cb. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 697.366135] nova-conductor[53809]: DEBUG nova.quota [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Getting quotas for user ce47ecda3b8a4a9ea5a5ce19137ac1cc and project 362a29354f714f0a8c002c5d933262cb. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 697.375449] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] [instance: 8fe2bccd-5b46-4067-b72b-bdbf726c0155] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 697.375566] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 697.376351] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 697.376351] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 697.387287] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] [instance: 8fe2bccd-5b46-4067-b72b-bdbf726c0155] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 697.388157] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 697.388332] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 697.388508] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 697.432643] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 697.433370] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 697.433370] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 697.621971] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Took 0.27 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 697.647838] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 697.648262] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 697.648455] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 697.753300] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 697.753300] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 697.753503] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 697.753830] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 697.754025] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 697.754225] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 697.764643] nova-conductor[53810]: DEBUG nova.quota [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Getting quotas for project 0e62d95c2bfa4690a3d6662a057b6cd2. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 697.771451] nova-conductor[53810]: DEBUG nova.quota [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Getting quotas for user b0ff0bbf7696481aa0a4d86c30d3dc7e and project 0e62d95c2bfa4690a3d6662a057b6cd2. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 697.783926] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] [instance: f347cbeb-a096-40db-8528-6cee24d390c2] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 697.784431] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 697.784653] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 697.784832] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 697.790544] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] [instance: f347cbeb-a096-40db-8528-6cee24d390c2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 697.790544] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 697.790544] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 697.790544] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 697.810543] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 697.810800] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 697.811018] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 698.669985] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] [instance: 8f553865-312c-4b6a-9383-43f2cbcd0b5e] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 661fd24c-c2b1-48c2-9bed-c6b679f2ea0a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 8f553865-312c-4b6a-9383-43f2cbcd0b5e was re-scheduled: Binding failed for port 661fd24c-c2b1-48c2-9bed-c6b679f2ea0a, please check neutron logs for more information.\n'] [ 698.670723] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 698.674144] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8f553865-312c-4b6a-9383-43f2cbcd0b5e.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8f553865-312c-4b6a-9383-43f2cbcd0b5e. [ 698.674536] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] [instance: 8f553865-312c-4b6a-9383-43f2cbcd0b5e] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8f553865-312c-4b6a-9383-43f2cbcd0b5e. [ 698.714694] nova-conductor[53809]: DEBUG nova.network.neutron [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] [instance: 8f553865-312c-4b6a-9383-43f2cbcd0b5e] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 698.780025] nova-conductor[53809]: DEBUG nova.network.neutron [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] [instance: 8f553865-312c-4b6a-9383-43f2cbcd0b5e] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 698.781622] nova-conductor[53809]: DEBUG nova.network.neutron [None req-c15b1af6-4519-4126-92ce-d46601dad842 tempest-ServersWithSpecificFlavorTestJSON-945038412 tempest-ServersWithSpecificFlavorTestJSON-945038412-project-member] [instance: 8f553865-312c-4b6a-9383-43f2cbcd0b5e] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 702.607823] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Took 0.20 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 702.628081] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 702.629899] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 702.633017] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 702.684806] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 702.685063] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 702.685421] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 702.685845] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 702.686045] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 702.686213] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 702.703838] nova-conductor[53810]: DEBUG nova.quota [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Getting quotas for project 362a29354f714f0a8c002c5d933262cb. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 702.710652] nova-conductor[53810]: DEBUG nova.quota [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Getting quotas for user ce47ecda3b8a4a9ea5a5ce19137ac1cc and project 362a29354f714f0a8c002c5d933262cb. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 702.717380] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] [instance: 4527b287-d099-443c-a424-185d02054be0] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 702.719173] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 702.719173] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 702.719173] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 702.726280] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] [instance: 4527b287-d099-443c-a424-185d02054be0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 702.726642] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 702.726863] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 702.727056] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 702.751627] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 702.751938] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 702.752186] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 704.101787] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] [instance: 4a2752b8-156a-4ec2-810c-3d6e7bc8554e] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 126cb36c-4690-4d88-bfab-0664961fe8ce, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 4a2752b8-156a-4ec2-810c-3d6e7bc8554e was re-scheduled: Binding failed for port 126cb36c-4690-4d88-bfab-0664961fe8ce, please check neutron logs for more information.\n'] [ 704.103204] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 704.103204] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4a2752b8-156a-4ec2-810c-3d6e7bc8554e.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4a2752b8-156a-4ec2-810c-3d6e7bc8554e. [ 704.103204] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] [instance: 4a2752b8-156a-4ec2-810c-3d6e7bc8554e] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4a2752b8-156a-4ec2-810c-3d6e7bc8554e. [ 704.135642] nova-conductor[53809]: DEBUG nova.network.neutron [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] [instance: 4a2752b8-156a-4ec2-810c-3d6e7bc8554e] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 704.223514] nova-conductor[53809]: DEBUG nova.network.neutron [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] [instance: 4a2752b8-156a-4ec2-810c-3d6e7bc8554e] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 704.227946] nova-conductor[53809]: DEBUG nova.network.neutron [None req-2d5bcf9a-ed4e-40a8-af8e-31ba992cabef tempest-ImagesNegativeTestJSON-1252542843 tempest-ImagesNegativeTestJSON-1252542843-project-member] [instance: 4a2752b8-156a-4ec2-810c-3d6e7bc8554e] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 704.459021] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Took 0.15 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 704.472955] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 704.472955] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 704.472955] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 704.522082] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 704.522549] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 704.522906] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 704.523472] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 704.523818] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 704.525101] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 704.540853] nova-conductor[53809]: DEBUG nova.quota [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Getting quotas for project 3e1db803f0ff4f29bb70e0a0d94c57e0. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 704.544801] nova-conductor[53809]: DEBUG nova.quota [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Getting quotas for user e64ca57e567146098521cd7356b9e3e2 and project 3e1db803f0ff4f29bb70e0a0d94c57e0. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 704.553648] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: ae2d23ed-62f5-48d5-bf8f-f5bbf38c9197] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 704.553648] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 704.553648] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 704.553648] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 704.560198] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: ae2d23ed-62f5-48d5-bf8f-f5bbf38c9197] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 704.560198] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 704.560198] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 704.560198] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 704.578414] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 704.578906] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 704.579697] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 705.392462] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Took 0.24 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 705.414406] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 705.414774] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 705.414990] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 705.473860] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 705.473860] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 705.473860] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 705.473860] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 705.473860] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 705.473860] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 705.486873] nova-conductor[53810]: DEBUG nova.quota [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Getting quotas for project e655d07206d94e0b8c65207d43edcdf4. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 705.491129] nova-conductor[53810]: DEBUG nova.quota [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Getting quotas for user b94cc1a2817f4ffcbabd083bca41bfc2 and project e655d07206d94e0b8c65207d43edcdf4. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 705.498243] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] [instance: de83001c-11a6-4c0d-86c9-9a5e582595bb] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 705.498622] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 705.498845] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 705.499137] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 705.503863] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] [instance: de83001c-11a6-4c0d-86c9-9a5e582595bb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 705.505172] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 705.505459] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 705.507220] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 705.554444] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 705.554812] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 705.554980] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 706.495181] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Took 0.28 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 706.518686] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 706.518803] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 706.519078] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 706.607746] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 706.608134] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 706.608941] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 706.608941] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 706.608941] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 706.609505] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 706.634601] nova-conductor[53810]: DEBUG nova.quota [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Getting quotas for project cd42a914c12b4f0c952fb8b911ea0e8a. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 706.641491] nova-conductor[53810]: DEBUG nova.quota [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Getting quotas for user f2681b9908124b12a9b39c688cf31f2a and project cd42a914c12b4f0c952fb8b911ea0e8a. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 706.651014] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] [instance: ec1394bf-266b-4830-8996-b6221c47c2e1] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 706.652284] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 706.652739] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 706.652739] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 706.673190] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] [instance: ec1394bf-266b-4830-8996-b6221c47c2e1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 706.674108] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 706.674388] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 706.674642] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 706.786141] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 706.786650] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 706.787024] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 707.012570] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] [instance: ad05c801-aff7-4a36-88e9-4994adfa3a8c] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 4fb5161b-a3fe-4af0-9a2c-e5502acf4bfd, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ad05c801-aff7-4a36-88e9-4994adfa3a8c was re-scheduled: Binding failed for port 4fb5161b-a3fe-4af0-9a2c-e5502acf4bfd, please check neutron logs for more information.\n'] [ 707.013936] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 707.014355] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ad05c801-aff7-4a36-88e9-4994adfa3a8c.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ad05c801-aff7-4a36-88e9-4994adfa3a8c. [ 707.015142] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] [instance: ad05c801-aff7-4a36-88e9-4994adfa3a8c] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ad05c801-aff7-4a36-88e9-4994adfa3a8c. [ 707.055577] nova-conductor[53810]: DEBUG nova.network.neutron [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] [instance: ad05c801-aff7-4a36-88e9-4994adfa3a8c] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 707.150347] nova-conductor[53810]: DEBUG nova.network.neutron [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] [instance: ad05c801-aff7-4a36-88e9-4994adfa3a8c] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 707.168561] nova-conductor[53810]: DEBUG nova.network.neutron [None req-ac788df8-e141-4402-aa21-8f517cf257a9 tempest-ServersTestJSON-1395993600 tempest-ServersTestJSON-1395993600-project-member] [instance: ad05c801-aff7-4a36-88e9-4994adfa3a8c] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 708.301916] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] [instance: f347cbeb-a096-40db-8528-6cee24d390c2] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port faf5d9cd-dd76-4f82-be96-e1413b39b9e5, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance f347cbeb-a096-40db-8528-6cee24d390c2 was re-scheduled: Binding failed for port faf5d9cd-dd76-4f82-be96-e1413b39b9e5, please check neutron logs for more information.\n'] [ 708.302553] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 708.302810] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f347cbeb-a096-40db-8528-6cee24d390c2.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f347cbeb-a096-40db-8528-6cee24d390c2. [ 708.303033] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] [instance: f347cbeb-a096-40db-8528-6cee24d390c2] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f347cbeb-a096-40db-8528-6cee24d390c2. [ 708.325305] nova-conductor[53810]: DEBUG nova.network.neutron [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] [instance: f347cbeb-a096-40db-8528-6cee24d390c2] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 708.398303] nova-conductor[53810]: DEBUG nova.network.neutron [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] [instance: f347cbeb-a096-40db-8528-6cee24d390c2] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 708.403730] nova-conductor[53810]: DEBUG nova.network.neutron [None req-32ffa299-2ab2-41ba-b128-78b5ab0cda25 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] [instance: f347cbeb-a096-40db-8528-6cee24d390c2] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 708.968918] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 708.990614] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 708.990614] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 708.990614] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 709.030317] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 709.030558] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 709.030735] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 709.031123] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 709.031317] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 709.031490] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 709.040775] nova-conductor[53810]: DEBUG nova.quota [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Getting quotas for project be173f6f52a14206b92e812f72c87d9b. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 709.044477] nova-conductor[53810]: DEBUG nova.quota [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Getting quotas for user d49d4ec89a0b466e8e292bc9bffc440f and project be173f6f52a14206b92e812f72c87d9b. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 709.050076] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] [instance: 3a0fe362-53a6-48a4-9fdd-cc81586bacfc] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 709.050510] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 709.050722] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 709.050895] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 709.055263] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] [instance: 3a0fe362-53a6-48a4-9fdd-cc81586bacfc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 709.055917] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 709.056141] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 709.056318] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 709.077411] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 709.077679] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 709.077862] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 714.026260] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 714.040286] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 714.040776] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 714.041173] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 714.087554] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 714.088041] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 714.088378] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 714.088911] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 714.089575] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 714.089920] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 714.102744] nova-conductor[53810]: DEBUG nova.quota [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Getting quotas for project 6e0140698aef43e4a744512d99c27d87. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 714.105862] nova-conductor[53810]: DEBUG nova.quota [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Getting quotas for user 495e4a012f524a31a7edbfc5c231eb5f and project 6e0140698aef43e4a744512d99c27d87. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 714.115852] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] [instance: 3495611c-05aa-4c1c-89fc-16ecb1fcb6bb] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 714.115852] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 714.115852] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 714.115852] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 714.125175] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] [instance: 3495611c-05aa-4c1c-89fc-16ecb1fcb6bb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 714.125175] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 714.125175] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 714.125175] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 714.143693] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 714.143693] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 714.143693] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 714.530008] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell0:SELECT=24 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 714.562538] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=108,nova_cell1:SAVEPOINT=9,nova_cell1:INSERT=238,nova_cell1:RELEASE=9 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 714.581941] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=159,nova_api:UPDATE=7,nova_api:DELETE=6 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 714.582622] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=129,nova_api:UPDATE=5,nova_api:DELETE=6 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 714.788694] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Took 0.22 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 714.807773] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 714.808247] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 714.809125] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 714.866892] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 714.866892] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 714.866892] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 714.867264] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 714.867455] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 714.867624] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 714.877063] nova-conductor[53810]: DEBUG nova.quota [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Getting quotas for project 16074166244d449b99488fc24f4f3d74. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 714.879474] nova-conductor[53810]: DEBUG nova.quota [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Getting quotas for user b861ada4972f4431b0b9bd46ae21f7cc and project 16074166244d449b99488fc24f4f3d74. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 714.885967] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: c9aebd15-4afa-47b4-9072-40ea371c2857] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 714.886504] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 714.886722] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 714.886898] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 714.899392] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: c9aebd15-4afa-47b4-9072-40ea371c2857] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 714.900161] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 714.900417] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 714.900602] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 714.924514] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 714.924514] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 714.924514] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 717.297337] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: ae2d23ed-62f5-48d5-bf8f-f5bbf38c9197] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 2fcf6fc6-cc25-4947-83e2-8491785cb7e2, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ae2d23ed-62f5-48d5-bf8f-f5bbf38c9197 was re-scheduled: Binding failed for port 2fcf6fc6-cc25-4947-83e2-8491785cb7e2, please check neutron logs for more information.\n'] [ 717.298150] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 717.298150] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ae2d23ed-62f5-48d5-bf8f-f5bbf38c9197.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ae2d23ed-62f5-48d5-bf8f-f5bbf38c9197. [ 717.298459] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: ae2d23ed-62f5-48d5-bf8f-f5bbf38c9197] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ae2d23ed-62f5-48d5-bf8f-f5bbf38c9197. [ 717.335166] nova-conductor[53810]: DEBUG nova.network.neutron [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: ae2d23ed-62f5-48d5-bf8f-f5bbf38c9197] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 717.493271] nova-conductor[53810]: DEBUG nova.network.neutron [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: ae2d23ed-62f5-48d5-bf8f-f5bbf38c9197] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 717.499615] nova-conductor[53810]: DEBUG nova.network.neutron [None req-568d335f-5ba7-48b9-b2ae-8950844d2be5 tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: ae2d23ed-62f5-48d5-bf8f-f5bbf38c9197] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 718.186579] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] [instance: de83001c-11a6-4c0d-86c9-9a5e582595bb] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 6c845135-77bc-40b8-ab5d-e008de23383a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance de83001c-11a6-4c0d-86c9-9a5e582595bb was re-scheduled: Binding failed for port 6c845135-77bc-40b8-ab5d-e008de23383a, please check neutron logs for more information.\n'] [ 718.189590] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 718.189590] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance de83001c-11a6-4c0d-86c9-9a5e582595bb.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance de83001c-11a6-4c0d-86c9-9a5e582595bb. [ 718.189590] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] [instance: de83001c-11a6-4c0d-86c9-9a5e582595bb] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance de83001c-11a6-4c0d-86c9-9a5e582595bb. [ 718.219454] nova-conductor[53810]: DEBUG nova.network.neutron [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] [instance: de83001c-11a6-4c0d-86c9-9a5e582595bb] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 718.320223] nova-conductor[53810]: DEBUG nova.network.neutron [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] [instance: de83001c-11a6-4c0d-86c9-9a5e582595bb] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 718.327882] nova-conductor[53810]: DEBUG nova.network.neutron [None req-939e2f71-1616-4b3f-beed-2129a71943a7 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150 tempest-FloatingIPsAssociationNegativeTestJSON-1058966150-project-member] [instance: de83001c-11a6-4c0d-86c9-9a5e582595bb] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 719.138123] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Took 0.18 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 719.172380] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 719.172626] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 719.172800] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 719.226350] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 719.226578] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 719.226750] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 719.227147] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 719.227336] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 719.227495] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 719.242449] nova-conductor[53809]: DEBUG nova.quota [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Getting quotas for project 0e62d95c2bfa4690a3d6662a057b6cd2. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 719.246249] nova-conductor[53809]: DEBUG nova.quota [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Getting quotas for user b0ff0bbf7696481aa0a4d86c30d3dc7e and project 0e62d95c2bfa4690a3d6662a057b6cd2. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 719.252928] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] [instance: b63e50ed-41f9-4e77-9b94-24915b296e2d] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 719.253485] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 719.253691] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 719.253860] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 719.262527] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] [instance: b63e50ed-41f9-4e77-9b94-24915b296e2d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 719.263473] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 719.263900] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 719.264102] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 719.285049] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] [instance: ec1394bf-266b-4830-8996-b6221c47c2e1] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 17a69317-ee7c-4a40-a06a-39537a1c4a67, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ec1394bf-266b-4830-8996-b6221c47c2e1 was re-scheduled: Binding failed for port 17a69317-ee7c-4a40-a06a-39537a1c4a67, please check neutron logs for more information.\n'] [ 719.287355] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 719.287355] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ec1394bf-266b-4830-8996-b6221c47c2e1.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ec1394bf-266b-4830-8996-b6221c47c2e1. [ 719.287355] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] [instance: ec1394bf-266b-4830-8996-b6221c47c2e1] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ec1394bf-266b-4830-8996-b6221c47c2e1. [ 719.303606] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 719.303606] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 719.303802] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 719.325703] nova-conductor[53809]: DEBUG nova.network.neutron [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] [instance: ec1394bf-266b-4830-8996-b6221c47c2e1] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 719.375034] nova-conductor[53809]: DEBUG nova.network.neutron [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] [instance: ec1394bf-266b-4830-8996-b6221c47c2e1] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 719.379322] nova-conductor[53809]: DEBUG nova.network.neutron [None req-3f94da63-dab4-43b5-8ccc-161a52557d23 tempest-AttachInterfacesV270Test-1607570909 tempest-AttachInterfacesV270Test-1607570909-project-member] [instance: ec1394bf-266b-4830-8996-b6221c47c2e1] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 722.790726] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] [instance: 3a0fe362-53a6-48a4-9fdd-cc81586bacfc] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 547f811a-085a-4dec-b008-82e96ca7b6be, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 3a0fe362-53a6-48a4-9fdd-cc81586bacfc was re-scheduled: Binding failed for port 547f811a-085a-4dec-b008-82e96ca7b6be, please check neutron logs for more information.\n'] [ 722.791080] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 722.791206] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3a0fe362-53a6-48a4-9fdd-cc81586bacfc.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3a0fe362-53a6-48a4-9fdd-cc81586bacfc. [ 722.791378] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] [instance: 3a0fe362-53a6-48a4-9fdd-cc81586bacfc] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3a0fe362-53a6-48a4-9fdd-cc81586bacfc. [ 722.815710] nova-conductor[53810]: DEBUG nova.network.neutron [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] [instance: 3a0fe362-53a6-48a4-9fdd-cc81586bacfc] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 722.966267] nova-conductor[53810]: DEBUG nova.network.neutron [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] [instance: 3a0fe362-53a6-48a4-9fdd-cc81586bacfc] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 722.970378] nova-conductor[53810]: DEBUG nova.network.neutron [None req-9f34a0ba-9d5a-413b-b9bd-87f36d39db09 tempest-ServerActionsTestOtherA-1328246412 tempest-ServerActionsTestOtherA-1328246412-project-member] [instance: 3a0fe362-53a6-48a4-9fdd-cc81586bacfc] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 724.121559] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Took 0.18 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 724.140064] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 724.141685] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 724.141685] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 724.194599] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 724.194957] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 724.195211] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 724.195647] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 724.195906] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 724.196150] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 724.214905] nova-conductor[53810]: DEBUG nova.quota [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Getting quotas for project ca364a2df93a424f8b66ee39d9b0b120. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 724.219088] nova-conductor[53810]: DEBUG nova.quota [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Getting quotas for user 2af2fd8431af45ca891f744f4d10b54f and project ca364a2df93a424f8b66ee39d9b0b120. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 724.231224] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: fda9c4e8-f38d-4b40-8678-5dc49bbf1ca4] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 724.231795] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 724.232076] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 724.232243] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 724.241993] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: fda9c4e8-f38d-4b40-8678-5dc49bbf1ca4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 724.243268] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 724.243268] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 724.243268] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 724.290522] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 724.293105] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 724.293105] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 727.866474] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Took 0.27 seconds to select destinations for 3 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 727.885776] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 727.886080] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 727.886312] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 727.933543] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 727.933971] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 727.934318] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 727.963559] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 727.963854] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 727.964087] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 727.995872] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 727.996199] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 727.996427] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 727.996860] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 727.997074] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 727.997248] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 728.009198] nova-conductor[53809]: DEBUG nova.quota [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Getting quotas for project dbc4594ef23b4419ac60a7a2195b2a94. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 728.012529] nova-conductor[53809]: DEBUG nova.quota [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Getting quotas for user 4d3dcf628fb041a2a1e7885e53c961b4 and project dbc4594ef23b4419ac60a7a2195b2a94. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 728.018731] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: 2f7123c7-f863-4bb6-a899-5feb618c6ce0] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 728.019331] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 728.019652] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 728.019879] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 728.024756] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: 2f7123c7-f863-4bb6-a899-5feb618c6ce0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 728.026528] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 728.026528] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 728.026528] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 728.046900] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 728.047383] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 728.047628] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 728.053778] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: 74e5ad9b-1b5b-492d-8642-ed271d8f70e3] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 728.054227] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 728.054484] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 728.054774] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 728.058241] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: 74e5ad9b-1b5b-492d-8642-ed271d8f70e3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 728.059176] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 728.059430] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 728.059647] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 728.079990] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 728.080376] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 728.080455] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 728.086446] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: d79dac42-38fa-401b-9864-5fbdf80b89ec] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 728.086997] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 728.087363] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 728.087781] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 728.091893] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: d79dac42-38fa-401b-9864-5fbdf80b89ec] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 728.092605] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 728.092858] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 728.093084] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 728.115123] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 728.115520] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 728.115752] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 728.675718] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Took 0.24 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 728.711679] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] [instance: b63e50ed-41f9-4e77-9b94-24915b296e2d] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port c142ff91-f8c7-47fa-849e-e56de72d907c, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance b63e50ed-41f9-4e77-9b94-24915b296e2d was re-scheduled: Binding failed for port c142ff91-f8c7-47fa-849e-e56de72d907c, please check neutron logs for more information.\n'] [ 728.711679] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 728.711679] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b63e50ed-41f9-4e77-9b94-24915b296e2d.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b63e50ed-41f9-4e77-9b94-24915b296e2d. [ 728.711679] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] [instance: b63e50ed-41f9-4e77-9b94-24915b296e2d] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b63e50ed-41f9-4e77-9b94-24915b296e2d. [ 728.752587] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 728.752906] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 728.753266] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 728.775401] nova-conductor[53810]: DEBUG nova.network.neutron [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] [instance: b63e50ed-41f9-4e77-9b94-24915b296e2d] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 728.809313] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 728.809545] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 728.809737] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 728.810134] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 728.810358] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 728.810655] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 728.827087] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] [instance: 3495611c-05aa-4c1c-89fc-16ecb1fcb6bb] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port c518959c-f1d5-4422-8c26-5d7f93544273, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 3495611c-05aa-4c1c-89fc-16ecb1fcb6bb was re-scheduled: Binding failed for port c518959c-f1d5-4422-8c26-5d7f93544273, please check neutron logs for more information.\n'] [ 728.827682] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 728.827925] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3495611c-05aa-4c1c-89fc-16ecb1fcb6bb.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3495611c-05aa-4c1c-89fc-16ecb1fcb6bb. [ 728.835735] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] [instance: 3495611c-05aa-4c1c-89fc-16ecb1fcb6bb] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3495611c-05aa-4c1c-89fc-16ecb1fcb6bb. [ 728.857293] nova-conductor[53809]: DEBUG nova.quota [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Getting quotas for project 1e76f73da19144f5b09ffee1f1d8853a. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 728.869782] nova-conductor[53809]: DEBUG nova.quota [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Getting quotas for user 034c0fef0ca64d55b894d99c202b880d and project 1e76f73da19144f5b09ffee1f1d8853a. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 728.891284] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] [instance: 2398e0a0-f13e-47e7-b735-906694ea4d58] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 728.892682] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 728.893585] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 728.893585] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 728.900632] nova-conductor[53809]: DEBUG nova.network.neutron [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] [instance: 3495611c-05aa-4c1c-89fc-16ecb1fcb6bb] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 728.904700] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] [instance: 2398e0a0-f13e-47e7-b735-906694ea4d58] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 728.905808] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 728.905808] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 728.905808] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 728.930325] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 728.930601] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 728.933061] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 728.970111] nova-conductor[53809]: DEBUG nova.network.neutron [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] [instance: 3495611c-05aa-4c1c-89fc-16ecb1fcb6bb] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 728.975169] nova-conductor[53809]: DEBUG nova.network.neutron [None req-8ced7a8c-1107-466d-bcc2-b14e2a058fa2 tempest-ServersTestManualDisk-714945568 tempest-ServersTestManualDisk-714945568-project-member] [instance: 3495611c-05aa-4c1c-89fc-16ecb1fcb6bb] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 729.134196] nova-conductor[53810]: DEBUG nova.network.neutron [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] [instance: b63e50ed-41f9-4e77-9b94-24915b296e2d] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 729.139696] nova-conductor[53810]: DEBUG nova.network.neutron [None req-37aff3b4-8eb2-4c6d-8ae0-6649f5a5f754 tempest-VolumesAdminNegativeTest-1485637538 tempest-VolumesAdminNegativeTest-1485637538-project-member] [instance: b63e50ed-41f9-4e77-9b94-24915b296e2d] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 730.972053] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: c9aebd15-4afa-47b4-9072-40ea371c2857] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 9713d50d-981c-43bd-92b0-7c766f373792, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance c9aebd15-4afa-47b4-9072-40ea371c2857 was re-scheduled: Binding failed for port 9713d50d-981c-43bd-92b0-7c766f373792, please check neutron logs for more information.\n'] [ 730.972677] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 730.972921] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c9aebd15-4afa-47b4-9072-40ea371c2857.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c9aebd15-4afa-47b4-9072-40ea371c2857. [ 730.973156] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: c9aebd15-4afa-47b4-9072-40ea371c2857] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c9aebd15-4afa-47b4-9072-40ea371c2857. [ 731.001582] nova-conductor[53810]: DEBUG nova.network.neutron [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: c9aebd15-4afa-47b4-9072-40ea371c2857] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 731.129191] nova-conductor[53810]: DEBUG nova.network.neutron [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: c9aebd15-4afa-47b4-9072-40ea371c2857] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 731.133472] nova-conductor[53810]: DEBUG nova.network.neutron [None req-8a009510-42db-4540-8479-8a6ed2a40dbd tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: c9aebd15-4afa-47b4-9072-40ea371c2857] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 732.724383] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Took 0.22 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 732.764532] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 732.764779] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 732.764957] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 732.820132] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 732.820132] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 732.820132] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 732.820132] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 732.820132] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 732.820132] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 732.830235] nova-conductor[53810]: DEBUG nova.quota [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Getting quotas for project 26026d6a7e084a0f81eba6e8b01d81d3. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 732.832066] nova-conductor[53810]: DEBUG nova.quota [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Getting quotas for user 02802ee5cbb14e6f9344e7479cfa2227 and project 26026d6a7e084a0f81eba6e8b01d81d3. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 732.841120] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] [instance: abdb8903-7574-40a4-ac7e-345b22fe1141] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 732.841816] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 732.841965] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 732.842110] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 732.854183] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] [instance: abdb8903-7574-40a4-ac7e-345b22fe1141] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 732.854183] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 732.854183] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 732.854183] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 732.883208] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 732.883208] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 732.883208] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 738.816869] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell0:SELECT=6 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 738.908352] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=41,nova_cell1:INSERT=97,nova_cell1:SAVEPOINT=2,nova_cell1:RELEASE=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 738.920475] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=38,nova_api:UPDATE=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 738.924034] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=50,nova_api:UPDATE=3,nova_api:DELETE=5 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 739.664351] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: fda9c4e8-f38d-4b40-8678-5dc49bbf1ca4] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 6cbd0454-8188-4cc6-859b-95c690163dd2, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance fda9c4e8-f38d-4b40-8678-5dc49bbf1ca4 was re-scheduled: Binding failed for port 6cbd0454-8188-4cc6-859b-95c690163dd2, please check neutron logs for more information.\n'] [ 739.664854] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 739.665112] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fda9c4e8-f38d-4b40-8678-5dc49bbf1ca4.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fda9c4e8-f38d-4b40-8678-5dc49bbf1ca4. [ 739.666201] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: fda9c4e8-f38d-4b40-8678-5dc49bbf1ca4] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fda9c4e8-f38d-4b40-8678-5dc49bbf1ca4. [ 739.704020] nova-conductor[53809]: DEBUG nova.network.neutron [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: fda9c4e8-f38d-4b40-8678-5dc49bbf1ca4] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 739.713188] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] [instance: a41cb33f-8340-4b15-b19d-0a7b9396eae7] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance a41cb33f-8340-4b15-b19d-0a7b9396eae7 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 739.713442] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 739.713549] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a41cb33f-8340-4b15-b19d-0a7b9396eae7.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a41cb33f-8340-4b15-b19d-0a7b9396eae7. [ 739.714262] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-f29cd538-c97a-42cc-81b2-bc5542e57d4e tempest-ServerShowV257Test-1759270808 tempest-ServerShowV257Test-1759270808-project-member] [instance: a41cb33f-8340-4b15-b19d-0a7b9396eae7] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a41cb33f-8340-4b15-b19d-0a7b9396eae7. [ 739.859174] nova-conductor[53809]: DEBUG nova.network.neutron [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: fda9c4e8-f38d-4b40-8678-5dc49bbf1ca4] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 739.867469] nova-conductor[53809]: DEBUG nova.network.neutron [None req-929cd172-2443-42b2-9540-44bbb5d2c2bb tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: fda9c4e8-f38d-4b40-8678-5dc49bbf1ca4] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 740.244650] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Took 0.36 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 740.260748] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 740.260994] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 740.261305] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 740.303813] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 740.303994] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 740.304253] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 740.304522] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 740.304716] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 740.304879] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 740.317301] nova-conductor[53809]: DEBUG nova.quota [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Getting quotas for project c85d469fb8e045f7b6981676c526d780. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 740.320418] nova-conductor[53809]: DEBUG nova.quota [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Getting quotas for user 2d0376ef3212459b883f3b757a17316f and project c85d469fb8e045f7b6981676c526d780. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 740.327735] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] [instance: 5bc95fc5-0d0d-4678-b2e6-1b7ca0e4b3d6] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 740.328266] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 740.328551] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 740.328734] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 740.333934] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] [instance: 5bc95fc5-0d0d-4678-b2e6-1b7ca0e4b3d6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 740.334600] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 740.335110] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 740.335110] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 740.355078] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 740.355327] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 740.355850] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 741.986536] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 742.002871] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 742.002871] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 742.002871] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 742.049402] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 742.049810] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 742.049976] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 742.050384] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 742.050584] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 742.050753] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 742.065746] nova-conductor[53809]: DEBUG nova.quota [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Getting quotas for project 16074166244d449b99488fc24f4f3d74. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 742.069783] nova-conductor[53809]: DEBUG nova.quota [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Getting quotas for user b861ada4972f4431b0b9bd46ae21f7cc and project 16074166244d449b99488fc24f4f3d74. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 742.076275] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 5188eb5e-7273-4fa9-bfe4-b1ec2bcdb3de] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 742.076786] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 742.076999] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 742.077216] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 742.083102] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 5188eb5e-7273-4fa9-bfe4-b1ec2bcdb3de] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 742.084218] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 742.084433] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 742.084576] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 742.104910] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 742.105191] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 742.105388] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 742.650073] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Took 0.22 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 742.672566] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 742.672824] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 742.673018] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 742.728589] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 742.728862] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 742.729135] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 742.729481] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 742.729691] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 742.729881] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 742.749846] nova-conductor[53809]: DEBUG nova.quota [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Getting quotas for project 660363d19fc745f6b616c24f0477a904. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 742.754343] nova-conductor[53809]: DEBUG nova.quota [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Getting quotas for user 389048589026452e88d501c50a137af0 and project 660363d19fc745f6b616c24f0477a904. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 742.761662] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] [instance: 42ca8a89-5938-491b-b122-deac71d18505] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 742.762206] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 742.762434] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 742.762686] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 742.767039] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] [instance: 42ca8a89-5938-491b-b122-deac71d18505] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 742.767770] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 742.767934] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 742.768124] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 742.808107] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 742.808107] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 742.808107] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 742.827602] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell0:SELECT=18 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 742.855473] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=81,nova_cell1:SAVEPOINT=8,nova_cell1:INSERT=180,nova_cell1:RELEASE=8 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 742.878787] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=110,nova_api:UPDATE=6,nova_api:DELETE=5 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 742.880035] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=106,nova_api:UPDATE=3,nova_api:DELETE=4 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 743.442094] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: 2f7123c7-f863-4bb6-a899-5feb618c6ce0] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port f7421b70-6d30-4120-be13-e6378ad0ffdb, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 2f7123c7-f863-4bb6-a899-5feb618c6ce0 was re-scheduled: Binding failed for port f7421b70-6d30-4120-be13-e6378ad0ffdb, please check neutron logs for more information.\n'] [ 743.443541] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 743.443541] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2f7123c7-f863-4bb6-a899-5feb618c6ce0.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2f7123c7-f863-4bb6-a899-5feb618c6ce0. [ 743.443541] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: 2f7123c7-f863-4bb6-a899-5feb618c6ce0] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2f7123c7-f863-4bb6-a899-5feb618c6ce0. [ 743.491413] nova-conductor[53810]: DEBUG nova.network.neutron [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: 2f7123c7-f863-4bb6-a899-5feb618c6ce0] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 743.581898] nova-conductor[53810]: DEBUG nova.network.neutron [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: 2f7123c7-f863-4bb6-a899-5feb618c6ce0] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 743.587238] nova-conductor[53810]: DEBUG nova.network.neutron [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: 2f7123c7-f863-4bb6-a899-5feb618c6ce0] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 745.100371] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: 74e5ad9b-1b5b-492d-8642-ed271d8f70e3] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 17e9cb25-c36e-4d26-b65a-e0135ece2194, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 74e5ad9b-1b5b-492d-8642-ed271d8f70e3 was re-scheduled: Binding failed for port 17e9cb25-c36e-4d26-b65a-e0135ece2194, please check neutron logs for more information.\n'] [ 745.101225] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 745.101577] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 74e5ad9b-1b5b-492d-8642-ed271d8f70e3.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 74e5ad9b-1b5b-492d-8642-ed271d8f70e3. [ 745.101917] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: 74e5ad9b-1b5b-492d-8642-ed271d8f70e3] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 74e5ad9b-1b5b-492d-8642-ed271d8f70e3. [ 745.128909] nova-conductor[53809]: DEBUG nova.network.neutron [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: 74e5ad9b-1b5b-492d-8642-ed271d8f70e3] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 745.337928] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=1020,nova_cell1:UPDATE=302,nova_cell1:INSERT=43,nova_cell1:SAVEPOINT=44,nova_cell1:RELEASE=44 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 745.442731] nova-conductor[53809]: DEBUG nova.network.neutron [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: 74e5ad9b-1b5b-492d-8642-ed271d8f70e3] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 745.449672] nova-conductor[53809]: DEBUG nova.network.neutron [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: 74e5ad9b-1b5b-492d-8642-ed271d8f70e3] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 745.525778] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: d79dac42-38fa-401b-9864-5fbdf80b89ec] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port e6d1e53a-74f0-4da3-8ede-885bbd0c7111, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance d79dac42-38fa-401b-9864-5fbdf80b89ec was re-scheduled: Binding failed for port e6d1e53a-74f0-4da3-8ede-885bbd0c7111, please check neutron logs for more information.\n'] [ 745.532019] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 745.532019] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d79dac42-38fa-401b-9864-5fbdf80b89ec.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d79dac42-38fa-401b-9864-5fbdf80b89ec. [ 745.532019] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: d79dac42-38fa-401b-9864-5fbdf80b89ec] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d79dac42-38fa-401b-9864-5fbdf80b89ec. [ 745.589316] nova-conductor[53810]: DEBUG nova.network.neutron [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: d79dac42-38fa-401b-9864-5fbdf80b89ec] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 745.720797] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Took 0.22 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 745.750026] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 745.750271] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 745.750451] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 745.802977] nova-conductor[53810]: DEBUG nova.network.neutron [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: d79dac42-38fa-401b-9864-5fbdf80b89ec] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 745.808339] nova-conductor[53810]: DEBUG nova.network.neutron [None req-c52a553a-2135-4689-9e47-2284985004d9 tempest-ListServersNegativeTestJSON-979928406 tempest-ListServersNegativeTestJSON-979928406-project-member] [instance: d79dac42-38fa-401b-9864-5fbdf80b89ec] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 745.810348] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 745.810563] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 745.810740] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 745.811165] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 745.811363] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 745.811541] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 745.822745] nova-conductor[53809]: DEBUG nova.quota [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Getting quotas for project 3e1db803f0ff4f29bb70e0a0d94c57e0. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 745.829764] nova-conductor[53809]: DEBUG nova.quota [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Getting quotas for user e64ca57e567146098521cd7356b9e3e2 and project 3e1db803f0ff4f29bb70e0a0d94c57e0. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 745.842260] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: 375d8f80-430a-4856-9b83-33c4aa945a57] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 745.842665] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 745.842881] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 745.843769] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 745.849524] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: 375d8f80-430a-4856-9b83-33c4aa945a57] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 745.850466] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 745.850785] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 745.851100] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 745.876110] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 745.876897] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 745.876897] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 746.964244] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] [instance: 2398e0a0-f13e-47e7-b735-906694ea4d58] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 905d20ea-65b5-4d7a-a080-9a1bfb47fb42, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 2398e0a0-f13e-47e7-b735-906694ea4d58 was re-scheduled: Binding failed for port 905d20ea-65b5-4d7a-a080-9a1bfb47fb42, please check neutron logs for more information.\n'] [ 746.965399] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 746.965660] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2398e0a0-f13e-47e7-b735-906694ea4d58.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2398e0a0-f13e-47e7-b735-906694ea4d58. [ 746.965975] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] [instance: 2398e0a0-f13e-47e7-b735-906694ea4d58] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2398e0a0-f13e-47e7-b735-906694ea4d58. [ 746.994390] nova-conductor[53809]: DEBUG nova.network.neutron [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] [instance: 2398e0a0-f13e-47e7-b735-906694ea4d58] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 747.088400] nova-conductor[53809]: DEBUG nova.network.neutron [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] [instance: 2398e0a0-f13e-47e7-b735-906694ea4d58] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 747.094020] nova-conductor[53809]: DEBUG nova.network.neutron [None req-61da2cdd-73df-4cd8-8666-6efb976502ee tempest-ServerActionsTestOtherB-1936320080 tempest-ServerActionsTestOtherB-1936320080-project-member] [instance: 2398e0a0-f13e-47e7-b735-906694ea4d58] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 748.447336] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=1030,nova_cell1:SAVEPOINT=60,nova_cell1:RELEASE=60,nova_cell1:UPDATE=282,nova_cell1:INSERT=46 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 750.805814] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] [instance: abdb8903-7574-40a4-ac7e-345b22fe1141] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 55d06218-1ea4-47f7-81e5-59f167d3ea24, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance abdb8903-7574-40a4-ac7e-345b22fe1141 was re-scheduled: Binding failed for port 55d06218-1ea4-47f7-81e5-59f167d3ea24, please check neutron logs for more information.\n'] [ 750.806447] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 750.806679] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance abdb8903-7574-40a4-ac7e-345b22fe1141.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance abdb8903-7574-40a4-ac7e-345b22fe1141. [ 750.806893] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] [instance: abdb8903-7574-40a4-ac7e-345b22fe1141] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance abdb8903-7574-40a4-ac7e-345b22fe1141. [ 750.837574] nova-conductor[53810]: DEBUG nova.network.neutron [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] [instance: abdb8903-7574-40a4-ac7e-345b22fe1141] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 750.956136] nova-conductor[53810]: DEBUG nova.network.neutron [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] [instance: abdb8903-7574-40a4-ac7e-345b22fe1141] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 750.965700] nova-conductor[53810]: DEBUG nova.network.neutron [None req-0c5ab126-e328-4542-864e-db8263bec43d tempest-ServerMetadataNegativeTestJSON-1006936469 tempest-ServerMetadataNegativeTestJSON-1006936469-project-member] [instance: abdb8903-7574-40a4-ac7e-345b22fe1141] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 751.669036] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Took 0.23 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 751.688649] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 751.688890] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 751.689082] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 751.750860] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 751.750860] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 751.750860] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 751.750860] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 751.750860] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 751.750860] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 751.764546] nova-conductor[53810]: DEBUG nova.quota [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Getting quotas for project ca364a2df93a424f8b66ee39d9b0b120. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 751.768440] nova-conductor[53810]: DEBUG nova.quota [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Getting quotas for user 2af2fd8431af45ca891f744f4d10b54f and project ca364a2df93a424f8b66ee39d9b0b120. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 751.776044] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 09300ec8-8a0e-4447-a32e-dd232e74fc53] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 751.776305] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 751.776530] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 751.776705] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 751.786016] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 09300ec8-8a0e-4447-a32e-dd232e74fc53] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 751.786016] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 751.786016] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 751.786016] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 751.815178] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 751.815178] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 751.815178] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 753.762644] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Took 0.22 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 753.780831] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 753.781098] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 753.781279] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 753.847703] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 753.847944] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 753.848644] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 753.848644] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 753.848765] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 753.848872] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 753.864039] nova-conductor[53809]: DEBUG nova.quota [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Getting quotas for project 5d52efc6161d484bbe7f70640caa646b. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 753.869434] nova-conductor[53809]: DEBUG nova.quota [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Getting quotas for user e8a7202adf9b42af989b3b02fcd12882 and project 5d52efc6161d484bbe7f70640caa646b. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 753.882090] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] [instance: bf9101c9-4072-4f72-8ac3-24b7a5b88b45] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 753.882090] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 753.882090] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 753.882090] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 753.892175] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] [instance: bf9101c9-4072-4f72-8ac3-24b7a5b88b45] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 753.893700] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 753.893700] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 753.893954] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 753.919044] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 753.919044] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 753.919044] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 756.839584] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Took 0.18 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 756.864863] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 756.865129] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 756.865565] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 756.929013] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 756.929281] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 756.929458] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 756.932729] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 756.932729] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 756.932729] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 756.941070] nova-conductor[53809]: DEBUG nova.quota [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Getting quotas for project abaf03622e1f42f485b74180094a7db2. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 756.944708] nova-conductor[53809]: DEBUG nova.quota [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Getting quotas for user 696ddf29d56e4b25820d33a408aa7397 and project abaf03622e1f42f485b74180094a7db2. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 756.951506] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] [instance: f6fba9ce-77a3-49cd-b52f-30d52e884a5b] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 756.952096] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 756.952341] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 756.952531] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 756.958427] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] [instance: f6fba9ce-77a3-49cd-b52f-30d52e884a5b] block_device_mapping [BlockDeviceMapping(attachment_id=c1d9d6e1-6b40-4fc7-bdd5-e222e4772ee4,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='d91ce8e0-5482-4bf8-9362-01c6b89272ee',volume_size=1,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 756.959282] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 756.959503] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 756.959679] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 756.980846] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 756.980973] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 756.981130] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 758.611200] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Took 0.21 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 758.656454] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 758.656584] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 758.656899] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 758.733217] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 758.733477] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 758.733650] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 758.734045] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 758.734275] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 758.734405] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 758.750028] nova-conductor[53810]: DEBUG nova.quota [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Getting quotas for project 7fd75a17bdcf4008a04e758009616713. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 758.754119] nova-conductor[53810]: DEBUG nova.quota [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Getting quotas for user f74f71de02b944e08a69a23f7fbfb0c3 and project 7fd75a17bdcf4008a04e758009616713. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 758.761815] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] [instance: 559acc57-5718-41bc-aa69-a8ca3272b28f] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 758.762357] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 758.762567] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 758.762737] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 758.769976] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] [instance: 559acc57-5718-41bc-aa69-a8ca3272b28f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 758.770695] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 758.770898] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 758.771079] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 758.792434] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 758.792672] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 758.792842] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 759.472283] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] [instance: 5bc95fc5-0d0d-4678-b2e6-1b7ca0e4b3d6] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 690df5ae-3e80-4edb-ac87-8f8780fda17c, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 5bc95fc5-0d0d-4678-b2e6-1b7ca0e4b3d6 was re-scheduled: Binding failed for port 690df5ae-3e80-4edb-ac87-8f8780fda17c, please check neutron logs for more information.\n'] [ 759.473384] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 759.473718] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5bc95fc5-0d0d-4678-b2e6-1b7ca0e4b3d6.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5bc95fc5-0d0d-4678-b2e6-1b7ca0e4b3d6. [ 759.474120] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] [instance: 5bc95fc5-0d0d-4678-b2e6-1b7ca0e4b3d6] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5bc95fc5-0d0d-4678-b2e6-1b7ca0e4b3d6. [ 759.512879] nova-conductor[53809]: DEBUG nova.network.neutron [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] [instance: 5bc95fc5-0d0d-4678-b2e6-1b7ca0e4b3d6] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 759.656722] nova-conductor[53809]: DEBUG nova.network.neutron [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] [instance: 5bc95fc5-0d0d-4678-b2e6-1b7ca0e4b3d6] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 759.665575] nova-conductor[53809]: DEBUG nova.network.neutron [None req-4b15fd02-16e9-4c7d-8ad4-9d16fc7cc909 tempest-SecurityGroupsTestJSON-969051472 tempest-SecurityGroupsTestJSON-969051472-project-member] [instance: 5bc95fc5-0d0d-4678-b2e6-1b7ca0e4b3d6] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 759.975282] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 5188eb5e-7273-4fa9-bfe4-b1ec2bcdb3de] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 6e291bb6-0f87-4960-b168-42bfea1ff96f, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 5188eb5e-7273-4fa9-bfe4-b1ec2bcdb3de was re-scheduled: Binding failed for port 6e291bb6-0f87-4960-b168-42bfea1ff96f, please check neutron logs for more information.\n'] [ 759.976256] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 759.976256] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5188eb5e-7273-4fa9-bfe4-b1ec2bcdb3de.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5188eb5e-7273-4fa9-bfe4-b1ec2bcdb3de. [ 759.976401] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 5188eb5e-7273-4fa9-bfe4-b1ec2bcdb3de] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5188eb5e-7273-4fa9-bfe4-b1ec2bcdb3de. [ 760.000025] nova-conductor[53809]: DEBUG nova.network.neutron [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 5188eb5e-7273-4fa9-bfe4-b1ec2bcdb3de] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 760.211368] nova-conductor[53809]: DEBUG nova.network.neutron [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 5188eb5e-7273-4fa9-bfe4-b1ec2bcdb3de] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 760.219050] nova-conductor[53809]: DEBUG nova.network.neutron [None req-82624f7b-68fd-458c-b5b5-b21356765a18 tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 5188eb5e-7273-4fa9-bfe4-b1ec2bcdb3de] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 760.585083] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Took 0.25 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 760.603886] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 760.604101] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 760.604251] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 760.705938] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 760.706381] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 760.710062] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 760.710062] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 760.710062] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 760.710062] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 760.721144] nova-conductor[53809]: DEBUG nova.quota [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Getting quotas for project 1f9b592f20764458954202cad4dcc1d1. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 760.726389] nova-conductor[53809]: DEBUG nova.quota [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Getting quotas for user 0ba2092a98474cd4bbd66180b95d3329 and project 1f9b592f20764458954202cad4dcc1d1. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 760.733266] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] [instance: 5f05e360-888d-41c3-87ec-e4838dc5a2f2] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 760.733779] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 760.734036] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 760.735440] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 760.749398] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] [instance: 5f05e360-888d-41c3-87ec-e4838dc5a2f2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 760.750208] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 760.750458] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 760.750649] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 760.784999] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 760.785305] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 760.785498] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 762.968062] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: 375d8f80-430a-4856-9b83-33c4aa945a57] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 09684df6-9453-4d3e-be24-48120b3abdab, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 375d8f80-430a-4856-9b83-33c4aa945a57 was re-scheduled: Binding failed for port 09684df6-9453-4d3e-be24-48120b3abdab, please check neutron logs for more information.\n'] [ 762.972293] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 762.972293] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 375d8f80-430a-4856-9b83-33c4aa945a57.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 375d8f80-430a-4856-9b83-33c4aa945a57. [ 762.972293] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: 375d8f80-430a-4856-9b83-33c4aa945a57] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 375d8f80-430a-4856-9b83-33c4aa945a57. [ 762.997932] nova-conductor[53809]: DEBUG nova.network.neutron [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: 375d8f80-430a-4856-9b83-33c4aa945a57] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 763.085582] nova-conductor[53809]: DEBUG nova.network.neutron [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: 375d8f80-430a-4856-9b83-33c4aa945a57] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 763.093720] nova-conductor[53809]: DEBUG nova.network.neutron [None req-90f653f5-0e51-46a4-9290-ad527dce5eed tempest-AttachInterfacesTestJSON-210578950 tempest-AttachInterfacesTestJSON-210578950-project-member] [instance: 375d8f80-430a-4856-9b83-33c4aa945a57] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 763.106196] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 763.122690] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 763.122931] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 763.123132] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 763.165720] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 763.165957] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 763.166148] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 763.166913] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 763.167151] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 763.167409] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 763.180918] nova-conductor[53810]: DEBUG nova.quota [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Getting quotas for project f86a86563cc047459d3e7c0553c82c63. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 763.184899] nova-conductor[53810]: DEBUG nova.quota [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Getting quotas for user c468b82bd9d64e19b419a393fff4af06 and project f86a86563cc047459d3e7c0553c82c63. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 763.191581] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 45a24553-3e31-4187-81fe-e9d1c8ca9353] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 763.192112] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 763.192345] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 763.192646] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 763.196199] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 45a24553-3e31-4187-81fe-e9d1c8ca9353] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 763.196859] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 763.197162] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 763.197411] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 763.220804] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 763.221074] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 763.221284] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 763.245175] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 763.282499] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 763.282853] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 763.283092] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 763.325024] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 763.325024] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 763.325024] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 763.325024] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 763.325024] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 763.325024] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 763.335982] nova-conductor[53809]: DEBUG nova.quota [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Getting quotas for project 0665c37087ef4f0abae09ed7fe33efae. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 763.339024] nova-conductor[53809]: DEBUG nova.quota [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Getting quotas for user 165cc88b32b74a0b9c4e62301b61da68 and project 0665c37087ef4f0abae09ed7fe33efae. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 763.345461] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] [instance: 0c167874-85c7-45bf-b296-971cdae1fe6a] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 763.345715] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 763.345984] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 763.346208] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 763.351377] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] [instance: 0c167874-85c7-45bf-b296-971cdae1fe6a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 763.352152] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 763.352496] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 763.352729] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 763.382632] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 763.383260] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 763.383260] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 767.299957] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 09300ec8-8a0e-4447-a32e-dd232e74fc53] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 1e9f8b2d-ce5e-4702-8c66-69e4a3ab9a16, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 09300ec8-8a0e-4447-a32e-dd232e74fc53 was re-scheduled: Binding failed for port 1e9f8b2d-ce5e-4702-8c66-69e4a3ab9a16, please check neutron logs for more information.\n'] [ 767.301273] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 767.301273] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 09300ec8-8a0e-4447-a32e-dd232e74fc53.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 09300ec8-8a0e-4447-a32e-dd232e74fc53. [ 767.301615] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 09300ec8-8a0e-4447-a32e-dd232e74fc53] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 09300ec8-8a0e-4447-a32e-dd232e74fc53. [ 767.334629] nova-conductor[53809]: DEBUG nova.network.neutron [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 09300ec8-8a0e-4447-a32e-dd232e74fc53] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 767.434488] nova-conductor[53809]: DEBUG nova.network.neutron [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 09300ec8-8a0e-4447-a32e-dd232e74fc53] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 767.444032] nova-conductor[53809]: DEBUG nova.network.neutron [None req-ae0b0106-6902-4dcd-acf8-d9157e0ec902 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 09300ec8-8a0e-4447-a32e-dd232e74fc53] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 771.997158] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 772.025692] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 772.025692] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 772.025692] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 772.066398] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 772.066398] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 772.066398] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 772.066398] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 772.066398] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 772.066398] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 772.081071] nova-conductor[53809]: DEBUG nova.quota [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Getting quotas for project 16074166244d449b99488fc24f4f3d74. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 772.084665] nova-conductor[53809]: DEBUG nova.quota [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Getting quotas for user b861ada4972f4431b0b9bd46ae21f7cc and project 16074166244d449b99488fc24f4f3d74. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 772.094277] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 8e57e090-2330-49f5-b939-be657884e506] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 772.094277] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 772.094522] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 772.094522] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 772.100428] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 8e57e090-2330-49f5-b939-be657884e506] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 772.101596] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 772.101596] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 772.101596] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 772.135628] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 772.136054] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 772.136369] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 773.173577] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell0:SELECT=6 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 773.199936] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=27,nova_cell1:INSERT=59,nova_cell1:SAVEPOINT=2,nova_cell1:RELEASE=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 773.216048] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=31,nova_api:UPDATE=3,nova_api:DELETE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 773.217607] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=41,nova_api:DELETE=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 773.710991] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] [instance: f6fba9ce-77a3-49cd-b52f-30d52e884a5b] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port abf3dd49-7566-41b8-9aec-33cee02d4bca, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance f6fba9ce-77a3-49cd-b52f-30d52e884a5b was re-scheduled: Binding failed for port abf3dd49-7566-41b8-9aec-33cee02d4bca, please check neutron logs for more information.\n'] [ 773.713951] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 773.714265] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f6fba9ce-77a3-49cd-b52f-30d52e884a5b.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f6fba9ce-77a3-49cd-b52f-30d52e884a5b. [ 773.714651] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] [instance: f6fba9ce-77a3-49cd-b52f-30d52e884a5b] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f6fba9ce-77a3-49cd-b52f-30d52e884a5b. [ 773.754094] nova-conductor[53810]: DEBUG nova.network.neutron [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] [instance: f6fba9ce-77a3-49cd-b52f-30d52e884a5b] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 773.830204] nova-conductor[53810]: DEBUG nova.network.neutron [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] [instance: f6fba9ce-77a3-49cd-b52f-30d52e884a5b] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 773.833630] nova-conductor[53810]: DEBUG nova.network.neutron [None req-f90f22bf-7498-4ad2-8bdc-513b3d956a88 tempest-ServersTestBootFromVolume-1411109147 tempest-ServersTestBootFromVolume-1411109147-project-member] [instance: f6fba9ce-77a3-49cd-b52f-30d52e884a5b] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 774.565755] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] [instance: 559acc57-5718-41bc-aa69-a8ca3272b28f] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port e094db18-1c04-4a5c-b1b3-977f24a0d11f, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 559acc57-5718-41bc-aa69-a8ca3272b28f was re-scheduled: Binding failed for port e094db18-1c04-4a5c-b1b3-977f24a0d11f, please check neutron logs for more information.\n'] [ 774.569741] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 774.570029] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 559acc57-5718-41bc-aa69-a8ca3272b28f.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 559acc57-5718-41bc-aa69-a8ca3272b28f. [ 774.570254] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] [instance: 559acc57-5718-41bc-aa69-a8ca3272b28f] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 559acc57-5718-41bc-aa69-a8ca3272b28f. [ 774.618935] nova-conductor[53810]: DEBUG nova.network.neutron [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] [instance: 559acc57-5718-41bc-aa69-a8ca3272b28f] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 774.666519] nova-conductor[53810]: DEBUG nova.network.neutron [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] [instance: 559acc57-5718-41bc-aa69-a8ca3272b28f] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 774.671761] nova-conductor[53810]: DEBUG nova.network.neutron [None req-c95bb8b6-21a7-446c-b23b-79e28e377fe9 tempest-InstanceActionsTestJSON-1957432545 tempest-InstanceActionsTestJSON-1957432545-project-member] [instance: 559acc57-5718-41bc-aa69-a8ca3272b28f] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 775.378327] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] [instance: 5f05e360-888d-41c3-87ec-e4838dc5a2f2] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port aa7b8e70-ec98-4d58-a0c7-7c7ea7927b18, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 5f05e360-888d-41c3-87ec-e4838dc5a2f2 was re-scheduled: Binding failed for port aa7b8e70-ec98-4d58-a0c7-7c7ea7927b18, please check neutron logs for more information.\n'] [ 775.378814] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 775.382890] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5f05e360-888d-41c3-87ec-e4838dc5a2f2.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5f05e360-888d-41c3-87ec-e4838dc5a2f2. [ 775.383150] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] [instance: 5f05e360-888d-41c3-87ec-e4838dc5a2f2] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5f05e360-888d-41c3-87ec-e4838dc5a2f2. [ 775.415028] nova-conductor[53810]: DEBUG nova.network.neutron [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] [instance: 5f05e360-888d-41c3-87ec-e4838dc5a2f2] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 775.469385] nova-conductor[53810]: DEBUG nova.network.neutron [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] [instance: 5f05e360-888d-41c3-87ec-e4838dc5a2f2] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 775.477200] nova-conductor[53810]: DEBUG nova.network.neutron [None req-728a5c1b-d451-40f7-959a-724e37ca2312 tempest-ServerAddressesNegativeTestJSON-605632328 tempest-ServerAddressesNegativeTestJSON-605632328-project-member] [instance: 5f05e360-888d-41c3-87ec-e4838dc5a2f2] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 777.167799] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 777.196824] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 777.197127] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 777.197343] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 777.256281] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 777.256866] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 777.257297] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 777.257786] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 777.257987] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 777.258166] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 777.289579] nova-conductor[53809]: DEBUG nova.quota [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Getting quotas for project ca364a2df93a424f8b66ee39d9b0b120. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 777.300098] nova-conductor[53809]: DEBUG nova.quota [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Getting quotas for user 2af2fd8431af45ca891f744f4d10b54f and project ca364a2df93a424f8b66ee39d9b0b120. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 777.315528] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 45a24553-3e31-4187-81fe-e9d1c8ca9353] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port be1fb9f7-258b-4aca-b899-f1818b72b1f4, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 45a24553-3e31-4187-81fe-e9d1c8ca9353 was re-scheduled: Binding failed for port be1fb9f7-258b-4aca-b899-f1818b72b1f4, please check neutron logs for more information.\n'] [ 777.316211] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 777.316488] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 45a24553-3e31-4187-81fe-e9d1c8ca9353.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 45a24553-3e31-4187-81fe-e9d1c8ca9353. [ 777.316857] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 45a24553-3e31-4187-81fe-e9d1c8ca9353] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 45a24553-3e31-4187-81fe-e9d1c8ca9353. [ 777.327190] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: c5e39fc1-989c-450b-9211-06cdd39700e6] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 777.327190] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 777.327190] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 777.327190] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 777.344696] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: c5e39fc1-989c-450b-9211-06cdd39700e6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 777.345398] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 777.345628] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 777.345807] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 777.379511] nova-conductor[53809]: DEBUG nova.network.neutron [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 45a24553-3e31-4187-81fe-e9d1c8ca9353] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 777.403267] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 777.403267] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 777.403267] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 777.449937] nova-conductor[53809]: DEBUG nova.network.neutron [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 45a24553-3e31-4187-81fe-e9d1c8ca9353] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 777.458607] nova-conductor[53809]: DEBUG nova.network.neutron [None req-e3a3c194-1149-4f8a-a621-f3803180dc97 tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 45a24553-3e31-4187-81fe-e9d1c8ca9353] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 777.472201] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] [instance: 0c167874-85c7-45bf-b296-971cdae1fe6a] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port d855f0dd-5b33-4e2d-8570-9f285221250a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 0c167874-85c7-45bf-b296-971cdae1fe6a was re-scheduled: Binding failed for port d855f0dd-5b33-4e2d-8570-9f285221250a, please check neutron logs for more information.\n'] [ 777.472881] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 777.476733] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0c167874-85c7-45bf-b296-971cdae1fe6a.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0c167874-85c7-45bf-b296-971cdae1fe6a. [ 777.476733] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] [instance: 0c167874-85c7-45bf-b296-971cdae1fe6a] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0c167874-85c7-45bf-b296-971cdae1fe6a. [ 777.532415] nova-conductor[53810]: DEBUG nova.network.neutron [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] [instance: 0c167874-85c7-45bf-b296-971cdae1fe6a] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 777.693803] nova-conductor[53810]: DEBUG nova.network.neutron [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] [instance: 0c167874-85c7-45bf-b296-971cdae1fe6a] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 777.700213] nova-conductor[53810]: DEBUG nova.network.neutron [None req-83dc816c-5698-40b0-b3e1-213289a4973e tempest-ServersNegativeTestJSON-1062426749 tempest-ServersNegativeTestJSON-1062426749-project-member] [instance: 0c167874-85c7-45bf-b296-971cdae1fe6a] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 781.175527] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Took 0.18 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 781.203667] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 781.203667] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 781.203667] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 781.265162] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 781.265409] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 781.265824] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 781.265921] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 781.266128] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 781.266321] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 781.277223] nova-conductor[53809]: DEBUG nova.quota [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Getting quotas for project d841db9575854aa388acc0bbb499fd52. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 781.285339] nova-conductor[53809]: DEBUG nova.quota [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Getting quotas for user 721ff5c63c5f405bb7be8c486b5fd162 and project d841db9575854aa388acc0bbb499fd52. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 781.291887] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: 93635056-506d-4672-ba46-d57e919df841] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 781.291947] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 781.292182] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 781.292437] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 781.299829] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: 93635056-506d-4672-ba46-d57e919df841] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 781.300528] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 781.300649] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 781.300840] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 781.324658] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 781.324924] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 781.325116] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 781.877687] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 781.892359] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 781.892620] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 781.892802] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 781.931409] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 781.931659] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 781.932364] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 781.932364] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 781.932509] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 781.932616] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 781.941459] nova-conductor[53809]: DEBUG nova.quota [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Getting quotas for project acc8187e781f41419aa48bdc98846e68. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 781.944065] nova-conductor[53809]: DEBUG nova.quota [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Getting quotas for user b15100e7cea84902abad60ded4d01420 and project acc8187e781f41419aa48bdc98846e68. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 781.950178] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] [instance: 170dbc1c-e9c8-4b23-a556-c6e212beda24] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 781.950688] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 781.950900] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 781.951087] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 781.957235] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] [instance: 170dbc1c-e9c8-4b23-a556-c6e212beda24] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 781.958534] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 781.958675] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 781.959055] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 781.976219] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 781.976449] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 781.976629] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 783.820717] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 783.839934] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 783.842549] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 783.842549] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 783.872295] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 8e57e090-2330-49f5-b939-be657884e506] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b9d0b31b-5bae-4e87-b3ee-21e3791aedd3, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 8e57e090-2330-49f5-b939-be657884e506 was re-scheduled: Binding failed for port b9d0b31b-5bae-4e87-b3ee-21e3791aedd3, please check neutron logs for more information.\n'] [ 783.872987] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 783.873295] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8e57e090-2330-49f5-b939-be657884e506.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8e57e090-2330-49f5-b939-be657884e506. [ 783.873645] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 8e57e090-2330-49f5-b939-be657884e506] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8e57e090-2330-49f5-b939-be657884e506. [ 783.917825] nova-conductor[53809]: DEBUG nova.network.neutron [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 8e57e090-2330-49f5-b939-be657884e506] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 783.920037] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 783.920374] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 783.920570] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 783.920949] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 783.921524] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 783.921722] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 783.933404] nova-conductor[53810]: DEBUG nova.quota [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Getting quotas for project 1ea05f21f7ce43ad9d22fa29068b4c8d. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 783.937145] nova-conductor[53810]: DEBUG nova.quota [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Getting quotas for user f0dc659b9ef04b0e9cdb22b584af3bbe and project 1ea05f21f7ce43ad9d22fa29068b4c8d. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 783.945607] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] [instance: 6f5a2f67-d112-4e97-bf4c-1ee5fea5c8b2] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 783.946750] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 783.946750] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 783.946750] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 783.950593] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] [instance: 6f5a2f67-d112-4e97-bf4c-1ee5fea5c8b2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 783.951378] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 783.951610] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 783.951799] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 783.973312] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 783.973442] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 783.973606] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 783.984987] nova-conductor[53809]: DEBUG nova.network.neutron [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 8e57e090-2330-49f5-b939-be657884e506] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 783.992786] nova-conductor[53809]: DEBUG nova.network.neutron [None req-8c3d28b1-ea37-4968-bb3a-afd25201211b tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 8e57e090-2330-49f5-b939-be657884e506] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 786.173809] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 786.188680] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 786.188961] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 786.189156] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 786.244150] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 786.244150] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 786.244150] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 786.244387] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 786.244602] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 786.244830] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 786.262021] nova-conductor[53809]: DEBUG nova.quota [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Getting quotas for project f86a86563cc047459d3e7c0553c82c63. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 786.263743] nova-conductor[53809]: DEBUG nova.quota [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Getting quotas for user c468b82bd9d64e19b419a393fff4af06 and project f86a86563cc047459d3e7c0553c82c63. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 786.273024] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 5f325b13-cd19-4cb1-a9f2-9d9e6942f9bd] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 786.273024] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 786.273024] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 786.273024] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 786.282050] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 5f325b13-cd19-4cb1-a9f2-9d9e6942f9bd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 786.282828] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 786.284579] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 786.284756] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 786.304931] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 786.305204] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 786.305436] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 786.798954] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: c5e39fc1-989c-450b-9211-06cdd39700e6] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 10e6e847-4841-4451-bb83-a18e31397cf3, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance c5e39fc1-989c-450b-9211-06cdd39700e6 was re-scheduled: Binding failed for port 10e6e847-4841-4451-bb83-a18e31397cf3, please check neutron logs for more information.\n'] [ 786.801023] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 786.801295] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c5e39fc1-989c-450b-9211-06cdd39700e6.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c5e39fc1-989c-450b-9211-06cdd39700e6. [ 786.802509] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: c5e39fc1-989c-450b-9211-06cdd39700e6] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c5e39fc1-989c-450b-9211-06cdd39700e6. [ 786.868333] nova-conductor[53809]: DEBUG nova.network.neutron [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: c5e39fc1-989c-450b-9211-06cdd39700e6] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 786.929507] nova-conductor[53809]: DEBUG nova.network.neutron [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: c5e39fc1-989c-450b-9211-06cdd39700e6] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 786.936587] nova-conductor[53809]: DEBUG nova.network.neutron [None req-126784f4-d1e9-48db-9ead-7b67112b03c7 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: c5e39fc1-989c-450b-9211-06cdd39700e6] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 788.842507] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Took 0.15 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 788.856813] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 788.857197] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 788.857419] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 788.927655] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 788.927900] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 788.928099] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 788.928510] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 788.928706] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 788.928874] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 788.937698] nova-conductor[53810]: DEBUG nova.quota [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Getting quotas for project b0b92a8e08a448da97a2428eee1d46b2. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 788.940589] nova-conductor[53810]: DEBUG nova.quota [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Getting quotas for user a264a55330124f6493ff80d4e18caf35 and project b0b92a8e08a448da97a2428eee1d46b2. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 788.948265] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] [instance: 405937e2-694a-4e50-bf6a-07e26193de16] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 788.948773] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 788.948967] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 788.949154] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 788.956547] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] [instance: 405937e2-694a-4e50-bf6a-07e26193de16] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 788.958207] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 788.958470] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 788.958660] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 788.981965] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 788.982213] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 788.982693] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 789.006621] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] [instance: 8fe2bccd-5b46-4067-b72b-bdbf726c0155] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 8fe2bccd-5b46-4067-b72b-bdbf726c0155 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 789.006621] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 789.006923] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8fe2bccd-5b46-4067-b72b-bdbf726c0155.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8fe2bccd-5b46-4067-b72b-bdbf726c0155. [ 789.007153] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-9de536da-f3b3-41b0-899c-b62f3494cb53 tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] [instance: 8fe2bccd-5b46-4067-b72b-bdbf726c0155] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8fe2bccd-5b46-4067-b72b-bdbf726c0155. [ 792.334207] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: 93635056-506d-4672-ba46-d57e919df841] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 6d2c454f-bf8b-4baf-9f24-ffd8a6c0609f, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 93635056-506d-4672-ba46-d57e919df841 was re-scheduled: Binding failed for port 6d2c454f-bf8b-4baf-9f24-ffd8a6c0609f, please check neutron logs for more information.\n'] [ 792.334207] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 792.334207] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 93635056-506d-4672-ba46-d57e919df841.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 93635056-506d-4672-ba46-d57e919df841. [ 792.334207] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: 93635056-506d-4672-ba46-d57e919df841] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 93635056-506d-4672-ba46-d57e919df841. [ 792.371163] nova-conductor[53809]: DEBUG nova.network.neutron [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: 93635056-506d-4672-ba46-d57e919df841] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 792.462400] nova-conductor[53809]: DEBUG nova.network.neutron [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: 93635056-506d-4672-ba46-d57e919df841] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 792.470811] nova-conductor[53809]: DEBUG nova.network.neutron [None req-b3970995-6fe7-42e5-a64c-f4753d03dcdf tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: 93635056-506d-4672-ba46-d57e919df841] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 793.155445] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 793.155754] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 793.155951] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 793.356979] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Took 0.18 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 793.381089] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 793.381089] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 793.381270] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 793.466038] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 793.466038] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 793.466038] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 793.466038] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 793.466038] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 793.466038] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 793.484414] nova-conductor[53809]: DEBUG nova.quota [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Getting quotas for project f777ddda84f140ab9ff2d486e9db78ca. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 793.494966] nova-conductor[53809]: DEBUG nova.quota [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Getting quotas for user 2711b8149e0f4877858ce3de2d3cbf50 and project f777ddda84f140ab9ff2d486e9db78ca. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 793.502725] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] [instance: 93ade29f-b55a-4a85-85dd-fc05699b3a21] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 793.503381] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 793.504601] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 793.504601] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 793.513952] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] [instance: 93ade29f-b55a-4a85-85dd-fc05699b3a21] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 793.514791] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 793.514946] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 793.515117] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 793.647018] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 793.647018] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 793.647018] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 794.952558] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 794.973273] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 794.973652] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 794.973917] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 795.040476] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 795.040745] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 795.040942] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 795.041346] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 795.041581] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 795.041765] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 795.057973] nova-conductor[53810]: DEBUG nova.quota [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Getting quotas for project 865f61767268462f999468ee9c444a6b. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 795.061868] nova-conductor[53810]: DEBUG nova.quota [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Getting quotas for user 6f5f04360628402f8040f2f71638f6ef and project 865f61767268462f999468ee9c444a6b. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 795.069434] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] [instance: ac936bda-d410-4437-866c-9b3a9e04e169] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 795.069988] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 795.070283] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 795.070512] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 795.078321] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] [instance: ac936bda-d410-4437-866c-9b3a9e04e169] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 795.079119] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 795.079971] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 795.079971] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 795.098185] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 795.098497] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 795.098744] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 795.490203] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Took 0.15 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 795.519088] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 795.519088] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 795.519088] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 795.584029] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 795.584029] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 795.584029] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 795.584331] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 795.584331] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 795.584546] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 795.602361] nova-conductor[53809]: DEBUG nova.quota [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Getting quotas for project ca364a2df93a424f8b66ee39d9b0b120. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 795.605743] nova-conductor[53809]: DEBUG nova.quota [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Getting quotas for user 2af2fd8431af45ca891f744f4d10b54f and project ca364a2df93a424f8b66ee39d9b0b120. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 795.615655] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 8ab0cb27-1972-4fcb-a549-442e720d872c] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 795.616257] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 795.616472] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 795.616640] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 795.626419] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 8ab0cb27-1972-4fcb-a549-442e720d872c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 795.627160] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 795.627377] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 795.627552] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 795.658263] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 795.658525] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 795.658702] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 796.509901] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 796.527218] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 796.527481] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 796.527773] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 796.582170] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 796.582520] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 796.582799] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 796.583488] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 796.583804] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 796.584100] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 796.598172] nova-conductor[53809]: DEBUG nova.quota [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Getting quotas for project 16074166244d449b99488fc24f4f3d74. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 796.601783] nova-conductor[53809]: DEBUG nova.quota [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Getting quotas for user b861ada4972f4431b0b9bd46ae21f7cc and project 16074166244d449b99488fc24f4f3d74. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 796.609332] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 974c8b62-f1ce-491a-a646-b091d3af2bb3] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 796.611150] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 796.611461] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 796.611667] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 796.626033] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 974c8b62-f1ce-491a-a646-b091d3af2bb3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 796.626826] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 796.627076] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 796.627273] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 796.660234] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 796.660533] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 796.660726] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 798.096282] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] [instance: 170dbc1c-e9c8-4b23-a556-c6e212beda24] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 09222fe3-a429-48de-868a-12985c3507fc, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 170dbc1c-e9c8-4b23-a556-c6e212beda24 was re-scheduled: Binding failed for port 09222fe3-a429-48de-868a-12985c3507fc, please check neutron logs for more information.\n'] [ 798.096282] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 798.096282] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 170dbc1c-e9c8-4b23-a556-c6e212beda24.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 170dbc1c-e9c8-4b23-a556-c6e212beda24. [ 798.096625] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] [instance: 170dbc1c-e9c8-4b23-a556-c6e212beda24] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 170dbc1c-e9c8-4b23-a556-c6e212beda24. [ 798.148465] nova-conductor[53810]: DEBUG nova.network.neutron [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] [instance: 170dbc1c-e9c8-4b23-a556-c6e212beda24] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 798.232111] nova-conductor[53810]: DEBUG nova.network.neutron [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] [instance: 170dbc1c-e9c8-4b23-a556-c6e212beda24] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 798.239199] nova-conductor[53810]: DEBUG nova.network.neutron [None req-80b94a18-8757-4aef-8b3e-952e53f2cae4 tempest-ServerActionsTestJSON-700045971 tempest-ServerActionsTestJSON-700045971-project-member] [instance: 170dbc1c-e9c8-4b23-a556-c6e212beda24] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 798.788851] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] [instance: 6f5a2f67-d112-4e97-bf4c-1ee5fea5c8b2] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 499fe493-f649-4f60-87ff-0031abbd78b7, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 6f5a2f67-d112-4e97-bf4c-1ee5fea5c8b2 was re-scheduled: Binding failed for port 499fe493-f649-4f60-87ff-0031abbd78b7, please check neutron logs for more information.\n'] [ 798.795015] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 798.795015] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6f5a2f67-d112-4e97-bf4c-1ee5fea5c8b2.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6f5a2f67-d112-4e97-bf4c-1ee5fea5c8b2. [ 798.795015] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] [instance: 6f5a2f67-d112-4e97-bf4c-1ee5fea5c8b2] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6f5a2f67-d112-4e97-bf4c-1ee5fea5c8b2. [ 798.823394] nova-conductor[53810]: DEBUG nova.network.neutron [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] [instance: 6f5a2f67-d112-4e97-bf4c-1ee5fea5c8b2] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 798.973832] nova-conductor[53810]: DEBUG nova.network.neutron [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] [instance: 6f5a2f67-d112-4e97-bf4c-1ee5fea5c8b2] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 798.979938] nova-conductor[53810]: DEBUG nova.network.neutron [None req-5e7d66fc-399a-45b5-85b0-741276e198e7 tempest-ServerDiagnosticsTest-672594855 tempest-ServerDiagnosticsTest-672594855-project-member] [instance: 6f5a2f67-d112-4e97-bf4c-1ee5fea5c8b2] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 799.438046] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=208,nova_api:UPDATE=7,nova_api:DELETE=10 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 799.438945] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=179,nova_api:UPDATE=9,nova_api:DELETE=6 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 799.611201] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 799.628126] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 799.629960] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 799.629960] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 799.632017] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=145,nova_cell1:INSERT=311,nova_cell1:SAVEPOINT=8,nova_cell1:RELEASE=8 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 799.670887] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 799.671135] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 799.671311] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 799.671681] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 799.671871] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 799.672047] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 799.676723] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell0:SELECT=33 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 799.685829] nova-conductor[53809]: DEBUG nova.quota [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Getting quotas for project d210a18ebdcd4c85a520ad1087664bea. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 799.689354] nova-conductor[53809]: DEBUG nova.quota [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Getting quotas for user 9d021d8b7cf444b19d9008f370d9f195 and project d210a18ebdcd4c85a520ad1087664bea. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 799.696267] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] [instance: 2e7a252e-071f-48bb-91ae-f4bdd7907059] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 799.696822] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 799.697060] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 799.697239] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 799.703618] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] [instance: 2e7a252e-071f-48bb-91ae-f4bdd7907059] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 799.704338] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 799.704564] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 799.704737] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 799.729803] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 799.729803] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 799.729803] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 800.106649] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 5f325b13-cd19-4cb1-a9f2-9d9e6942f9bd] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port ce73ef47-1792-432a-9ea6-cc8a5709ee84, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 5f325b13-cd19-4cb1-a9f2-9d9e6942f9bd was re-scheduled: Binding failed for port ce73ef47-1792-432a-9ea6-cc8a5709ee84, please check neutron logs for more information.\n'] [ 800.107391] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 800.108143] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5f325b13-cd19-4cb1-a9f2-9d9e6942f9bd.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5f325b13-cd19-4cb1-a9f2-9d9e6942f9bd. [ 800.108422] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 5f325b13-cd19-4cb1-a9f2-9d9e6942f9bd] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5f325b13-cd19-4cb1-a9f2-9d9e6942f9bd. [ 800.137856] nova-conductor[53810]: DEBUG nova.network.neutron [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 5f325b13-cd19-4cb1-a9f2-9d9e6942f9bd] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 800.229044] nova-conductor[53810]: DEBUG nova.network.neutron [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 5f325b13-cd19-4cb1-a9f2-9d9e6942f9bd] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 800.235056] nova-conductor[53810]: DEBUG nova.network.neutron [None req-87a4f0fb-7aad-40f2-85d1-9b3f95f9975e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 5f325b13-cd19-4cb1-a9f2-9d9e6942f9bd] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 803.315352] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] [instance: 405937e2-694a-4e50-bf6a-07e26193de16] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 4a5edbdb-c6ce-48e6-bb8e-3f88f5501ddc, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 405937e2-694a-4e50-bf6a-07e26193de16 was re-scheduled: Binding failed for port 4a5edbdb-c6ce-48e6-bb8e-3f88f5501ddc, please check neutron logs for more information.\n'] [ 803.316381] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 803.316724] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 405937e2-694a-4e50-bf6a-07e26193de16.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 405937e2-694a-4e50-bf6a-07e26193de16. [ 803.317089] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] [instance: 405937e2-694a-4e50-bf6a-07e26193de16] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 405937e2-694a-4e50-bf6a-07e26193de16. [ 803.346905] nova-conductor[53810]: DEBUG nova.network.neutron [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] [instance: 405937e2-694a-4e50-bf6a-07e26193de16] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 803.467328] nova-conductor[53810]: DEBUG nova.network.neutron [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] [instance: 405937e2-694a-4e50-bf6a-07e26193de16] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 803.475094] nova-conductor[53810]: DEBUG nova.network.neutron [None req-91dd8bef-65a0-451f-9338-59381431f1ea tempest-ServersNegativeTestMultiTenantJSON-2038123386 tempest-ServersNegativeTestMultiTenantJSON-2038123386-project-member] [instance: 405937e2-694a-4e50-bf6a-07e26193de16] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 805.050409] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell0:SELECT=6 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 805.082713] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=27,nova_cell1:SAVEPOINT=3,nova_cell1:INSERT=60,nova_cell1:RELEASE=3 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 805.093138] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=39,nova_api:DELETE=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 805.098325] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=35,nova_api:UPDATE=3,nova_api:DELETE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 805.788567] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=940,nova_cell1:SAVEPOINT=50,nova_cell1:RELEASE=50,nova_cell1:UPDATE=281,nova_cell1:INSERT=44,nova_cell1:DELETE=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 807.042562] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] [instance: 93ade29f-b55a-4a85-85dd-fc05699b3a21] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port e9d024bf-bebd-413f-98c5-7f94c6c1dfef, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 93ade29f-b55a-4a85-85dd-fc05699b3a21 was re-scheduled: Binding failed for port e9d024bf-bebd-413f-98c5-7f94c6c1dfef, please check neutron logs for more information.\n'] [ 807.043506] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 807.043727] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 93ade29f-b55a-4a85-85dd-fc05699b3a21.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 93ade29f-b55a-4a85-85dd-fc05699b3a21. [ 807.043946] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] [instance: 93ade29f-b55a-4a85-85dd-fc05699b3a21] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 93ade29f-b55a-4a85-85dd-fc05699b3a21. [ 807.088613] nova-conductor[53810]: DEBUG nova.network.neutron [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] [instance: 93ade29f-b55a-4a85-85dd-fc05699b3a21] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 807.109051] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Took 0.20 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 807.134225] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 807.134466] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 807.135044] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 807.171991] nova-conductor[53810]: DEBUG nova.network.neutron [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] [instance: 93ade29f-b55a-4a85-85dd-fc05699b3a21] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 807.178288] nova-conductor[53810]: DEBUG nova.network.neutron [None req-8d42a19c-6e52-47d3-aed2-87814b4d4804 tempest-ServerGroupTestJSON-1745706281 tempest-ServerGroupTestJSON-1745706281-project-member] [instance: 93ade29f-b55a-4a85-85dd-fc05699b3a21] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 807.213771] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 807.213999] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 807.214187] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 807.214559] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 807.215018] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 807.215018] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 807.251821] nova-conductor[53810]: DEBUG nova.quota [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Getting quotas for project 2011298d12d046aebc57dc28f99c9cc6. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 807.259314] nova-conductor[53810]: DEBUG nova.quota [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Getting quotas for user bea60c029be640338b24207747cf2c13 and project 2011298d12d046aebc57dc28f99c9cc6. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 807.266794] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] [instance: c30b7561-2c76-4d68-93da-f73e7c6a0ed7] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 807.267914] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 807.267914] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 807.267914] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 807.272973] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] [instance: c30b7561-2c76-4d68-93da-f73e7c6a0ed7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 807.273706] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 807.273938] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 807.274127] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 807.301194] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 807.301388] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 807.301668] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 807.697221] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] [instance: ac936bda-d410-4437-866c-9b3a9e04e169] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port f18a0e9f-1571-438a-8dba-e8294e47767b, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ac936bda-d410-4437-866c-9b3a9e04e169 was re-scheduled: Binding failed for port f18a0e9f-1571-438a-8dba-e8294e47767b, please check neutron logs for more information.\n'] [ 807.697221] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 807.698249] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ac936bda-d410-4437-866c-9b3a9e04e169.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ac936bda-d410-4437-866c-9b3a9e04e169. [ 807.698249] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] [instance: ac936bda-d410-4437-866c-9b3a9e04e169] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ac936bda-d410-4437-866c-9b3a9e04e169. [ 807.738351] nova-conductor[53809]: DEBUG nova.network.neutron [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] [instance: ac936bda-d410-4437-866c-9b3a9e04e169] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 807.828480] nova-conductor[53809]: DEBUG nova.network.neutron [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] [instance: ac936bda-d410-4437-866c-9b3a9e04e169] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 807.835068] nova-conductor[53809]: DEBUG nova.network.neutron [None req-909192ed-311c-4224-b124-0feb2c19c2da tempest-ServerAddressesTestJSON-1402003061 tempest-ServerAddressesTestJSON-1402003061-project-member] [instance: ac936bda-d410-4437-866c-9b3a9e04e169] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 808.022570] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Took 0.22 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 808.050514] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 808.050779] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 808.050898] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 808.103037] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 808.103037] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 808.103037] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 808.103037] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 808.103037] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 808.103037] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 808.112616] nova-conductor[53810]: DEBUG nova.quota [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Getting quotas for project 23a10ccee87e4d6a9959b310e8a58d86. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 808.116896] nova-conductor[53810]: DEBUG nova.quota [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Getting quotas for user 58092638c8144ee383fc39898d1291af and project 23a10ccee87e4d6a9959b310e8a58d86. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 808.127390] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] [instance: 025992d2-423e-492d-b820-7d9f3554c0f6] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 808.127390] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 808.127390] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 808.127390] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 808.135633] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] [instance: 025992d2-423e-492d-b820-7d9f3554c0f6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 808.136281] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 808.136515] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 808.136693] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 808.157434] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 808.157613] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 808.157788] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 808.397032] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 8ab0cb27-1972-4fcb-a549-442e720d872c] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 5db53ed0-af24-4bcb-b026-cc45252750ed, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 8ab0cb27-1972-4fcb-a549-442e720d872c was re-scheduled: Binding failed for port 5db53ed0-af24-4bcb-b026-cc45252750ed, please check neutron logs for more information.\n'] [ 808.397032] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 808.397032] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8ab0cb27-1972-4fcb-a549-442e720d872c.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8ab0cb27-1972-4fcb-a549-442e720d872c. [ 808.397032] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 8ab0cb27-1972-4fcb-a549-442e720d872c] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8ab0cb27-1972-4fcb-a549-442e720d872c. [ 808.433870] nova-conductor[53809]: DEBUG nova.network.neutron [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 8ab0cb27-1972-4fcb-a549-442e720d872c] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 808.477530] nova-conductor[53809]: DEBUG nova.network.neutron [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 8ab0cb27-1972-4fcb-a549-442e720d872c] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 808.480636] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=1000,nova_cell1:SAVEPOINT=55,nova_cell1:RELEASE=55,nova_cell1:UPDATE=283,nova_cell1:INSERT=40 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 808.486064] nova-conductor[53809]: DEBUG nova.network.neutron [None req-f76b41af-70d3-47af-9aad-0218502ebc35 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 8ab0cb27-1972-4fcb-a549-442e720d872c] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 809.309703] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Took 0.18 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 809.328928] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 809.329181] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 809.329354] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 809.392058] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 809.392427] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 809.392732] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 809.393238] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 809.393238] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 809.393624] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 809.412937] nova-conductor[53810]: DEBUG nova.quota [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Getting quotas for project 9a13458250fe40d78872c7d479b16b73. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 809.419096] nova-conductor[53810]: DEBUG nova.quota [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Getting quotas for user 5e7f8b3201474ff28bf523b845f0bd87 and project 9a13458250fe40d78872c7d479b16b73. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 809.430930] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] [instance: de136287-7dfb-4829-aef7-96ae82acfa65] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 809.431503] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 809.431720] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 809.431895] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 809.447191] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] [instance: de136287-7dfb-4829-aef7-96ae82acfa65] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 809.447982] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 809.448321] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 809.448408] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 809.474582] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 809.474848] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 809.475089] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 809.678906] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell0:SELECT=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 809.708022] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=8,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=20,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 809.725904] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 809.726586] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=13,nova_api:DELETE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 810.191552] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] [instance: 2e7a252e-071f-48bb-91ae-f4bdd7907059] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port ace1f76f-cb81-4243-85fd-09aa9ab57d92, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 2e7a252e-071f-48bb-91ae-f4bdd7907059 was re-scheduled: Binding failed for port ace1f76f-cb81-4243-85fd-09aa9ab57d92, please check neutron logs for more information.\n'] [ 810.192186] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 810.192534] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2e7a252e-071f-48bb-91ae-f4bdd7907059.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2e7a252e-071f-48bb-91ae-f4bdd7907059. [ 810.192816] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] [instance: 2e7a252e-071f-48bb-91ae-f4bdd7907059] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2e7a252e-071f-48bb-91ae-f4bdd7907059. [ 810.230294] nova-conductor[53810]: DEBUG nova.network.neutron [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] [instance: 2e7a252e-071f-48bb-91ae-f4bdd7907059] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 810.307757] nova-conductor[53810]: DEBUG nova.network.neutron [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] [instance: 2e7a252e-071f-48bb-91ae-f4bdd7907059] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 810.314939] nova-conductor[53810]: DEBUG nova.network.neutron [None req-a5c8cd26-f0b6-4ca4-b85f-007ffbf842ce tempest-InstanceActionsV221TestJSON-346798859 tempest-InstanceActionsV221TestJSON-346798859-project-member] [instance: 2e7a252e-071f-48bb-91ae-f4bdd7907059] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 810.554568] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 974c8b62-f1ce-491a-a646-b091d3af2bb3] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 552cc4f1-2181-4a39-8014-88c9feb747d9, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 974c8b62-f1ce-491a-a646-b091d3af2bb3 was re-scheduled: Binding failed for port 552cc4f1-2181-4a39-8014-88c9feb747d9, please check neutron logs for more information.\n'] [ 810.555586] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 810.555586] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 974c8b62-f1ce-491a-a646-b091d3af2bb3.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 974c8b62-f1ce-491a-a646-b091d3af2bb3. [ 810.555729] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 974c8b62-f1ce-491a-a646-b091d3af2bb3] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 974c8b62-f1ce-491a-a646-b091d3af2bb3. [ 810.604097] nova-conductor[53810]: DEBUG nova.network.neutron [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 974c8b62-f1ce-491a-a646-b091d3af2bb3] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 810.639362] nova-conductor[53810]: DEBUG nova.network.neutron [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 974c8b62-f1ce-491a-a646-b091d3af2bb3] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 810.643191] nova-conductor[53810]: DEBUG nova.network.neutron [None req-ff715d4f-057b-4e47-b11f-77766dc24abc tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 974c8b62-f1ce-491a-a646-b091d3af2bb3] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 810.704570] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Took 0.18 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 810.725806] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 810.727893] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 810.727893] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 810.787085] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 810.787301] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 810.787473] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 810.787931] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 810.788093] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 810.788277] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 810.801246] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Took 0.24 seconds to select destinations for 2 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 810.802767] nova-conductor[53810]: DEBUG nova.quota [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Getting quotas for project 9bf4f2e40b3744e68d4740ffe6a94004. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 810.808030] nova-conductor[53810]: DEBUG nova.quota [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Getting quotas for user 6caf6ee2fa984516b47e691afad810b2 and project 9bf4f2e40b3744e68d4740ffe6a94004. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 810.815808] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] [instance: 282ecbf4-cd05-4ea0-bb3f-708969856b7e] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 810.816328] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 810.816568] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 810.817267] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 810.817430] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 810.817672] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 810.817849] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 810.825816] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] [instance: 282ecbf4-cd05-4ea0-bb3f-708969856b7e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 810.825816] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 810.825816] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 810.825816] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 810.846043] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 810.846302] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 810.846731] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 810.855089] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 810.855337] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 810.855519] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 810.899969] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 810.900268] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 810.900447] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 810.900831] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 810.901118] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 810.901309] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 810.922450] nova-conductor[53809]: DEBUG nova.quota [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Getting quotas for project 278860d9a5104cebabf418408d8558d2. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 810.923539] nova-conductor[53809]: DEBUG nova.quota [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Getting quotas for user 919a60d5a73a4077b510084cc28a9499 and project 278860d9a5104cebabf418408d8558d2. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 810.932424] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: eec2b74f-9b6c-4566-a0dc-da1fe9578715] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 810.933332] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 810.933332] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 810.933332] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 810.936514] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: eec2b74f-9b6c-4566-a0dc-da1fe9578715] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 810.938045] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 810.938045] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 810.938045] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 810.953290] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 810.953543] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 810.953714] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 810.964683] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: cfda19ad-e831-4018-9228-e96fede0bae6] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 810.964683] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 810.964683] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 810.964683] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 810.970628] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: cfda19ad-e831-4018-9228-e96fede0bae6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 810.971336] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 810.971557] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 810.971733] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 810.999600] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 810.999928] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 811.000096] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 811.305309] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Took 0.14 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 811.326393] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 811.326674] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 811.326749] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 811.380781] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 811.381075] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 811.381299] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 811.381711] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 811.381943] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 811.382163] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 811.409292] nova-conductor[53810]: DEBUG nova.quota [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Getting quotas for project c3bb640c7d6845c3ae230cb286d1a8ba. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 811.415606] nova-conductor[53810]: DEBUG nova.quota [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Getting quotas for user eac14fe0d2c5455cb602e436721b2054 and project c3bb640c7d6845c3ae230cb286d1a8ba. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 811.440962] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] [instance: 0bbf5ae6-c31d-4ea2-8581-6bed6c11c2d4] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 811.441509] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 811.441729] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 811.441907] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 811.466944] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] [instance: 0bbf5ae6-c31d-4ea2-8581-6bed6c11c2d4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 811.467363] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 811.467587] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 811.467765] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 811.499088] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 811.499581] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 811.499581] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 812.210673] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 812.233906] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 812.234155] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 812.234330] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 812.322064] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 812.322064] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 812.322064] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 812.322476] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 812.323173] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 812.323398] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 812.334884] nova-conductor[53809]: DEBUG nova.quota [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Getting quotas for project f86a86563cc047459d3e7c0553c82c63. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 812.337651] nova-conductor[53809]: DEBUG nova.quota [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Getting quotas for user c468b82bd9d64e19b419a393fff4af06 and project f86a86563cc047459d3e7c0553c82c63. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 812.347135] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: b779ccfb-9278-4a00-aadf-bd4afb0ab54a] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 812.348049] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 812.348049] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 812.348049] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 812.358399] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: b779ccfb-9278-4a00-aadf-bd4afb0ab54a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 812.359174] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 812.359398] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 812.359577] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 812.392227] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 812.392227] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 812.392446] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 815.153375] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 815.165110] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 815.165110] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 815.165528] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 815.219913] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 815.222088] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 815.222088] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 815.222088] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 815.222088] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 815.222088] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 815.234700] nova-conductor[53810]: DEBUG nova.quota [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Getting quotas for project d841db9575854aa388acc0bbb499fd52. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 815.243141] nova-conductor[53810]: DEBUG nova.quota [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Getting quotas for user 721ff5c63c5f405bb7be8c486b5fd162 and project d841db9575854aa388acc0bbb499fd52. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 815.258202] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: 50266004-15d2-46cf-9f48-315f24831d24] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 815.258202] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 815.258202] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 815.258202] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 815.268322] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: 50266004-15d2-46cf-9f48-315f24831d24] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 815.271341] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 815.271341] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 815.271341] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 815.300448] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 815.300664] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 815.300861] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 817.568219] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] [instance: c30b7561-2c76-4d68-93da-f73e7c6a0ed7] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 81aad203-f976-4c1c-81d7-6848338446d7, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance c30b7561-2c76-4d68-93da-f73e7c6a0ed7 was re-scheduled: Binding failed for port 81aad203-f976-4c1c-81d7-6848338446d7, please check neutron logs for more information.\n'] [ 817.568219] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 817.568219] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c30b7561-2c76-4d68-93da-f73e7c6a0ed7.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c30b7561-2c76-4d68-93da-f73e7c6a0ed7. [ 817.568219] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] [instance: c30b7561-2c76-4d68-93da-f73e7c6a0ed7] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c30b7561-2c76-4d68-93da-f73e7c6a0ed7. [ 817.607864] nova-conductor[53809]: DEBUG nova.network.neutron [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] [instance: c30b7561-2c76-4d68-93da-f73e7c6a0ed7] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 817.644443] nova-conductor[53809]: DEBUG nova.network.neutron [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] [instance: c30b7561-2c76-4d68-93da-f73e7c6a0ed7] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 817.648142] nova-conductor[53809]: DEBUG nova.network.neutron [None req-4887a2ec-4ebb-4d88-930a-8c9f780af24e tempest-FloatingIPsAssociationTestJSON-1026406162 tempest-FloatingIPsAssociationTestJSON-1026406162-project-member] [instance: c30b7561-2c76-4d68-93da-f73e7c6a0ed7] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 818.737503] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] [instance: 025992d2-423e-492d-b820-7d9f3554c0f6] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 5a95018b-84db-45e5-8b22-d05ea50159a5, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 025992d2-423e-492d-b820-7d9f3554c0f6 was re-scheduled: Binding failed for port 5a95018b-84db-45e5-8b22-d05ea50159a5, please check neutron logs for more information.\n'] [ 818.737503] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 818.737503] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 025992d2-423e-492d-b820-7d9f3554c0f6.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 025992d2-423e-492d-b820-7d9f3554c0f6. [ 818.738247] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] [instance: 025992d2-423e-492d-b820-7d9f3554c0f6] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 025992d2-423e-492d-b820-7d9f3554c0f6. [ 818.768862] nova-conductor[53809]: DEBUG nova.network.neutron [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] [instance: 025992d2-423e-492d-b820-7d9f3554c0f6] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 818.834330] nova-conductor[53809]: DEBUG nova.network.neutron [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] [instance: 025992d2-423e-492d-b820-7d9f3554c0f6] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 818.837952] nova-conductor[53809]: DEBUG nova.network.neutron [None req-1c57db5c-9254-46dc-9e36-e447e34ffafc tempest-AttachInterfacesUnderV243Test-2070080807 tempest-AttachInterfacesUnderV243Test-2070080807-project-member] [instance: 025992d2-423e-492d-b820-7d9f3554c0f6] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 820.345409] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 820.365824] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 820.365824] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 820.365824] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 820.436160] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 820.438210] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 820.438210] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 820.438210] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 820.438210] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 820.438210] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 820.451036] nova-conductor[53809]: DEBUG nova.quota [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Getting quotas for project ca364a2df93a424f8b66ee39d9b0b120. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 820.455147] nova-conductor[53809]: DEBUG nova.quota [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Getting quotas for user 2af2fd8431af45ca891f744f4d10b54f and project ca364a2df93a424f8b66ee39d9b0b120. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 820.463206] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: da0610f3-ab6b-4496-ba18-2794869a2831] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 820.463882] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 820.464243] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 820.464734] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 820.468786] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: da0610f3-ab6b-4496-ba18-2794869a2831] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 820.469662] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 820.470157] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 820.470506] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 820.480503] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] [instance: de136287-7dfb-4829-aef7-96ae82acfa65] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b63cbd7f-55d4-4445-bc1f-668af5b37db5, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance de136287-7dfb-4829-aef7-96ae82acfa65 was re-scheduled: Binding failed for port b63cbd7f-55d4-4445-bc1f-668af5b37db5, please check neutron logs for more information.\n'] [ 820.481140] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 820.481522] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance de136287-7dfb-4829-aef7-96ae82acfa65.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance de136287-7dfb-4829-aef7-96ae82acfa65. [ 820.485156] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] [instance: de136287-7dfb-4829-aef7-96ae82acfa65] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance de136287-7dfb-4829-aef7-96ae82acfa65. [ 820.503799] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 820.504329] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 820.504839] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 820.540640] nova-conductor[53810]: DEBUG nova.network.neutron [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] [instance: de136287-7dfb-4829-aef7-96ae82acfa65] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 820.654098] nova-conductor[53810]: DEBUG nova.network.neutron [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] [instance: de136287-7dfb-4829-aef7-96ae82acfa65] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 820.660537] nova-conductor[53810]: DEBUG nova.network.neutron [None req-25dae54b-4644-4825-9a67-0cbe718eea52 tempest-ServersV294TestFqdnHostnames-1074575199 tempest-ServersV294TestFqdnHostnames-1074575199-project-member] [instance: de136287-7dfb-4829-aef7-96ae82acfa65] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 821.683870] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: cfda19ad-e831-4018-9228-e96fede0bae6] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 72f3e9d6-cf65-454e-b7a9-2f0d35c86416, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance cfda19ad-e831-4018-9228-e96fede0bae6 was re-scheduled: Binding failed for port 72f3e9d6-cf65-454e-b7a9-2f0d35c86416, please check neutron logs for more information.\n'] [ 821.684713] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 821.685048] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance cfda19ad-e831-4018-9228-e96fede0bae6.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance cfda19ad-e831-4018-9228-e96fede0bae6. [ 821.685275] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: cfda19ad-e831-4018-9228-e96fede0bae6] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance cfda19ad-e831-4018-9228-e96fede0bae6. [ 821.713030] nova-conductor[53810]: DEBUG nova.network.neutron [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: cfda19ad-e831-4018-9228-e96fede0bae6] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 821.812104] nova-conductor[53810]: DEBUG nova.network.neutron [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: cfda19ad-e831-4018-9228-e96fede0bae6] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 821.816826] nova-conductor[53810]: DEBUG nova.network.neutron [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: cfda19ad-e831-4018-9228-e96fede0bae6] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 822.872575] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 822.886868] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 822.887126] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 822.887308] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 822.928075] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 822.928330] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 822.928507] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 822.928899] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 822.929101] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 822.929271] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 822.939690] nova-conductor[53810]: DEBUG nova.quota [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Getting quotas for project 16074166244d449b99488fc24f4f3d74. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 822.943259] nova-conductor[53810]: DEBUG nova.quota [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Getting quotas for user b861ada4972f4431b0b9bd46ae21f7cc and project 16074166244d449b99488fc24f4f3d74. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 822.949332] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 7c3017dc-afd5-438e-ae23-ca0d7d4c01af] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 822.949800] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 822.950024] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 822.950210] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 822.955513] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 7c3017dc-afd5-438e-ae23-ca0d7d4c01af] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 822.956209] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 822.956419] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 822.956591] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 822.973436] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 822.973672] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 822.973847] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 825.730522] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] [instance: 282ecbf4-cd05-4ea0-bb3f-708969856b7e] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 650ee4d0-5f8e-44be-b3d7-48701ce29ed7, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 282ecbf4-cd05-4ea0-bb3f-708969856b7e was re-scheduled: Binding failed for port 650ee4d0-5f8e-44be-b3d7-48701ce29ed7, please check neutron logs for more information.\n'] [ 825.731648] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 825.732064] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 282ecbf4-cd05-4ea0-bb3f-708969856b7e.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 282ecbf4-cd05-4ea0-bb3f-708969856b7e. [ 825.733155] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] [instance: 282ecbf4-cd05-4ea0-bb3f-708969856b7e] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 282ecbf4-cd05-4ea0-bb3f-708969856b7e. [ 825.765491] nova-conductor[53809]: DEBUG nova.network.neutron [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] [instance: 282ecbf4-cd05-4ea0-bb3f-708969856b7e] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 825.915125] nova-conductor[53809]: DEBUG nova.network.neutron [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] [instance: 282ecbf4-cd05-4ea0-bb3f-708969856b7e] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 825.919954] nova-conductor[53809]: DEBUG nova.network.neutron [None req-8456ed09-09bb-4c88-a761-169e4179e901 tempest-ServerRescueTestJSONUnderV235-809091443 tempest-ServerRescueTestJSONUnderV235-809091443-project-member] [instance: 282ecbf4-cd05-4ea0-bb3f-708969856b7e] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 826.582090] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: eec2b74f-9b6c-4566-a0dc-da1fe9578715] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 410abb97-ddc2-44bc-8edb-f7e053fb7273, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance eec2b74f-9b6c-4566-a0dc-da1fe9578715 was re-scheduled: Binding failed for port 410abb97-ddc2-44bc-8edb-f7e053fb7273, please check neutron logs for more information.\n'] [ 826.585021] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 826.585021] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance eec2b74f-9b6c-4566-a0dc-da1fe9578715.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance eec2b74f-9b6c-4566-a0dc-da1fe9578715. [ 826.585021] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: eec2b74f-9b6c-4566-a0dc-da1fe9578715] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance eec2b74f-9b6c-4566-a0dc-da1fe9578715. [ 826.660174] nova-conductor[53809]: DEBUG nova.network.neutron [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: eec2b74f-9b6c-4566-a0dc-da1fe9578715] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 826.761694] nova-conductor[53809]: DEBUG nova.network.neutron [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: eec2b74f-9b6c-4566-a0dc-da1fe9578715] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 826.770860] nova-conductor[53809]: DEBUG nova.network.neutron [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: eec2b74f-9b6c-4566-a0dc-da1fe9578715] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 826.785100] nova-conductor[53809]: DEBUG nova.db.main.api [None req-256876ce-f95d-4e2a-9b31-55c62edbd9ad tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Created instance_extra for eec2b74f-9b6c-4566-a0dc-da1fe9578715 {{(pid=53809) instance_extra_update_by_uuid /opt/stack/nova/nova/db/main/api.py:2551}} [ 827.061089] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] [instance: 0bbf5ae6-c31d-4ea2-8581-6bed6c11c2d4] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port def2fbbb-a0c8-4947-8402-02a1fc787cff, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 0bbf5ae6-c31d-4ea2-8581-6bed6c11c2d4 was re-scheduled: Binding failed for port def2fbbb-a0c8-4947-8402-02a1fc787cff, please check neutron logs for more information.\n'] [ 827.061794] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 827.062197] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0bbf5ae6-c31d-4ea2-8581-6bed6c11c2d4.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0bbf5ae6-c31d-4ea2-8581-6bed6c11c2d4. [ 827.062403] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] [instance: 0bbf5ae6-c31d-4ea2-8581-6bed6c11c2d4] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0bbf5ae6-c31d-4ea2-8581-6bed6c11c2d4. [ 827.097090] nova-conductor[53810]: DEBUG nova.network.neutron [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] [instance: 0bbf5ae6-c31d-4ea2-8581-6bed6c11c2d4] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 827.161980] nova-conductor[53810]: DEBUG nova.network.neutron [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] [instance: 0bbf5ae6-c31d-4ea2-8581-6bed6c11c2d4] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 827.166614] nova-conductor[53810]: DEBUG nova.network.neutron [None req-7f9b5c6e-cb7d-43d8-9e77-39cb639cc993 tempest-ServerPasswordTestJSON-1865709812 tempest-ServerPasswordTestJSON-1865709812-project-member] [instance: 0bbf5ae6-c31d-4ea2-8581-6bed6c11c2d4] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 830.441779] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell0:SELECT=6 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 830.473416] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=34,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=77,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 830.491714] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=42,nova_api:UPDATE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 830.498310] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=38,nova_api:UPDATE=3,nova_api:DELETE=4 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 832.006905] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: b779ccfb-9278-4a00-aadf-bd4afb0ab54a] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 28cf4c9f-fbd9-40b9-8aaf-326c2dc4a433, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance b779ccfb-9278-4a00-aadf-bd4afb0ab54a was re-scheduled: Binding failed for port 28cf4c9f-fbd9-40b9-8aaf-326c2dc4a433, please check neutron logs for more information.\n'] [ 832.007389] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 832.007595] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b779ccfb-9278-4a00-aadf-bd4afb0ab54a.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b779ccfb-9278-4a00-aadf-bd4afb0ab54a. [ 832.007816] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: b779ccfb-9278-4a00-aadf-bd4afb0ab54a] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b779ccfb-9278-4a00-aadf-bd4afb0ab54a. [ 832.047955] nova-conductor[53809]: DEBUG nova.network.neutron [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: b779ccfb-9278-4a00-aadf-bd4afb0ab54a] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 832.172854] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Took 0.21 seconds to select destinations for 2 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 832.180839] nova-conductor[53809]: DEBUG nova.network.neutron [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: b779ccfb-9278-4a00-aadf-bd4afb0ab54a] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 832.186566] nova-conductor[53809]: DEBUG nova.network.neutron [None req-2052ebab-d6dc-4f80-8812-ebfc42cea74e tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: b779ccfb-9278-4a00-aadf-bd4afb0ab54a] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 832.197640] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 832.198140] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 832.198140] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 832.237856] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 832.238095] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 832.238276] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 832.277860] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 832.279218] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 832.279218] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 832.279218] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 832.279218] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 832.279218] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 832.293174] nova-conductor[53810]: DEBUG nova.quota [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Getting quotas for project 278860d9a5104cebabf418408d8558d2. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 832.300935] nova-conductor[53810]: DEBUG nova.quota [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Getting quotas for user 919a60d5a73a4077b510084cc28a9499 and project 278860d9a5104cebabf418408d8558d2. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 832.311764] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: a7144e03-407c-436c-9e22-1eeaeb43a210] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 832.312272] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 832.312562] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 832.313039] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 832.323338] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: a7144e03-407c-436c-9e22-1eeaeb43a210] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 832.324196] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 832.324441] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 832.324637] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 832.341736] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 832.341973] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 832.342162] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 832.352942] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: 2d8f035e-8a57-4937-a394-10d94a5630ad] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 832.352942] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 832.352942] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 832.352942] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 832.356348] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: 2d8f035e-8a57-4937-a394-10d94a5630ad] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 832.357047] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 832.357267] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 832.357441] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 832.375214] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 832.375477] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 832.375638] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 833.115594] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 833.132937] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 833.133180] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 833.133364] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 833.196207] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 833.196572] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 833.196572] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 833.196896] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 833.197124] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 833.197605] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 833.229219] nova-conductor[53810]: DEBUG nova.quota [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Getting quotas for project 3bab637ad53d433db6fb2017b6c0c2aa. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 833.234480] nova-conductor[53810]: DEBUG nova.quota [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Getting quotas for user 66924fb566fb455bada0c920137fb884 and project 3bab637ad53d433db6fb2017b6c0c2aa. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 833.259318] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] [instance: 53400571-766c-4020-b163-87a8816199cd] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 833.259755] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 833.260092] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 833.260300] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 833.272712] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] [instance: 53400571-766c-4020-b163-87a8816199cd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 833.273402] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 833.273999] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 833.273999] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 833.301845] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 833.302123] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 833.302324] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 834.857461] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 834.880389] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 834.880389] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 834.880389] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 834.923310] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 834.923963] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 834.924087] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 834.924462] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 834.924659] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 834.924828] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 834.936144] nova-conductor[53810]: DEBUG nova.quota [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Getting quotas for project 063546fa9ba840158e484a974edfd79a. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 834.940048] nova-conductor[53810]: DEBUG nova.quota [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Getting quotas for user 027b7525da4940fe841f1f585b852b04 and project 063546fa9ba840158e484a974edfd79a. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 834.946110] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] [instance: a0be018c-59be-4eb0-a3aa-b7814ce240e8] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 834.948405] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 834.948405] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 834.948405] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 834.953190] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] [instance: 4527b287-d099-443c-a424-185d02054be0] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 4527b287-d099-443c-a424-185d02054be0 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 834.954281] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 834.957229] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4527b287-d099-443c-a424-185d02054be0.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4527b287-d099-443c-a424-185d02054be0. [ 834.957704] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-2cef5af9-8695-4847-a9ac-663a1cce5d4f tempest-ServerShowV247Test-985300212 tempest-ServerShowV247Test-985300212-project-member] [instance: 4527b287-d099-443c-a424-185d02054be0] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4527b287-d099-443c-a424-185d02054be0. [ 834.971226] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] [instance: a0be018c-59be-4eb0-a3aa-b7814ce240e8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 834.972136] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 834.972227] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 834.972444] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 835.007537] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 835.007782] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 835.008097] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 835.759406] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 835.774233] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 835.774364] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 835.775336] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 835.822793] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 835.826019] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 835.826019] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 835.826019] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 835.826019] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 835.826019] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 835.835299] nova-conductor[53810]: DEBUG nova.quota [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Getting quotas for project f86a86563cc047459d3e7c0553c82c63. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 835.837704] nova-conductor[53810]: DEBUG nova.quota [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Getting quotas for user c468b82bd9d64e19b419a393fff4af06 and project f86a86563cc047459d3e7c0553c82c63. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 835.846054] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 95a16efa-3240-4e15-ae19-03aaef61e2de] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 835.846451] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 835.846666] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 835.846838] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 835.854872] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 95a16efa-3240-4e15-ae19-03aaef61e2de] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 835.855692] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 835.856037] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 835.856277] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 835.895581] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 835.896153] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 835.896153] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 836.044976] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 836.061882] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 836.062400] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 836.062684] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 836.114651] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 836.114914] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 836.115118] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 836.115510] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 836.115699] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 836.115861] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 836.130758] nova-conductor[53810]: DEBUG nova.quota [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Getting quotas for project 063546fa9ba840158e484a974edfd79a. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 836.134183] nova-conductor[53810]: DEBUG nova.quota [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Getting quotas for user 027b7525da4940fe841f1f585b852b04 and project 063546fa9ba840158e484a974edfd79a. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 836.139927] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] [instance: 6288806c-d634-4749-8538-7188954788f0] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 836.140555] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 836.140715] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 836.140895] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 836.144793] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] [instance: 6288806c-d634-4749-8538-7188954788f0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 836.145576] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 836.145651] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 836.145863] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 836.162731] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 836.163030] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 836.164303] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 836.605394] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: 50266004-15d2-46cf-9f48-315f24831d24] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 7284a82c-18a5-43a0-b4d9-67251e9955b7, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 50266004-15d2-46cf-9f48-315f24831d24 was re-scheduled: Binding failed for port 7284a82c-18a5-43a0-b4d9-67251e9955b7, please check neutron logs for more information.\n'] [ 836.605394] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 836.605738] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 50266004-15d2-46cf-9f48-315f24831d24.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 50266004-15d2-46cf-9f48-315f24831d24. [ 836.607066] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: 50266004-15d2-46cf-9f48-315f24831d24] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 50266004-15d2-46cf-9f48-315f24831d24. [ 836.650746] nova-conductor[53810]: DEBUG nova.network.neutron [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: 50266004-15d2-46cf-9f48-315f24831d24] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 836.774106] nova-conductor[53810]: DEBUG nova.network.neutron [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: 50266004-15d2-46cf-9f48-315f24831d24] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 836.777646] nova-conductor[53810]: DEBUG nova.network.neutron [None req-abf94687-081c-4338-9ec4-e622a3c61eb0 tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: 50266004-15d2-46cf-9f48-315f24831d24] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 839.350996] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: da0610f3-ab6b-4496-ba18-2794869a2831] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 9c39d9bd-de1a-462d-9218-e51ebfb16c02, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance da0610f3-ab6b-4496-ba18-2794869a2831 was re-scheduled: Binding failed for port 9c39d9bd-de1a-462d-9218-e51ebfb16c02, please check neutron logs for more information.\n'] [ 839.350996] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 839.350996] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance da0610f3-ab6b-4496-ba18-2794869a2831.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance da0610f3-ab6b-4496-ba18-2794869a2831. [ 839.350996] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: da0610f3-ab6b-4496-ba18-2794869a2831] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance da0610f3-ab6b-4496-ba18-2794869a2831. [ 839.396385] nova-conductor[53810]: DEBUG nova.network.neutron [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: da0610f3-ab6b-4496-ba18-2794869a2831] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 839.536065] nova-conductor[53810]: DEBUG nova.network.neutron [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: da0610f3-ab6b-4496-ba18-2794869a2831] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 839.537036] nova-conductor[53810]: DEBUG nova.network.neutron [None req-b5a007fe-b64a-46f0-ac66-8994d058b753 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: da0610f3-ab6b-4496-ba18-2794869a2831] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 843.245979] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 7c3017dc-afd5-438e-ae23-ca0d7d4c01af] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port c11db4be-ceba-4564-a97e-7f4fc2d8afa7, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 7c3017dc-afd5-438e-ae23-ca0d7d4c01af was re-scheduled: Binding failed for port c11db4be-ceba-4564-a97e-7f4fc2d8afa7, please check neutron logs for more information.\n'] [ 843.247037] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 843.247037] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7c3017dc-afd5-438e-ae23-ca0d7d4c01af.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7c3017dc-afd5-438e-ae23-ca0d7d4c01af. [ 843.247913] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 7c3017dc-afd5-438e-ae23-ca0d7d4c01af] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7c3017dc-afd5-438e-ae23-ca0d7d4c01af. [ 843.283053] nova-conductor[53810]: DEBUG nova.network.neutron [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 7c3017dc-afd5-438e-ae23-ca0d7d4c01af] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 843.410178] nova-conductor[53810]: DEBUG nova.network.neutron [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 7c3017dc-afd5-438e-ae23-ca0d7d4c01af] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 843.417437] nova-conductor[53810]: DEBUG nova.network.neutron [None req-34c11c68-0316-4863-b52d-c2ed6796f28d tempest-ServerDiskConfigTestJSON-1897992284 tempest-ServerDiskConfigTestJSON-1897992284-project-member] [instance: 7c3017dc-afd5-438e-ae23-ca0d7d4c01af] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 844.516931] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: a7144e03-407c-436c-9e22-1eeaeb43a210] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 408f5613-8187-43fd-9e67-c27b5776065e, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance a7144e03-407c-436c-9e22-1eeaeb43a210 was re-scheduled: Binding failed for port 408f5613-8187-43fd-9e67-c27b5776065e, please check neutron logs for more information.\n'] [ 844.516931] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 844.518627] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a7144e03-407c-436c-9e22-1eeaeb43a210.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a7144e03-407c-436c-9e22-1eeaeb43a210. [ 844.518931] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: a7144e03-407c-436c-9e22-1eeaeb43a210] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a7144e03-407c-436c-9e22-1eeaeb43a210. [ 844.550185] nova-conductor[53809]: DEBUG nova.network.neutron [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: a7144e03-407c-436c-9e22-1eeaeb43a210] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 844.626196] nova-conductor[53809]: DEBUG nova.network.neutron [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: a7144e03-407c-436c-9e22-1eeaeb43a210] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 844.630202] nova-conductor[53809]: DEBUG nova.network.neutron [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: a7144e03-407c-436c-9e22-1eeaeb43a210] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 845.129814] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Took 0.23 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 845.144994] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 845.145258] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 845.145435] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 845.204011] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 845.204011] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 845.204011] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 845.204011] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 845.204011] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 845.204011] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 845.220603] nova-conductor[53809]: DEBUG nova.quota [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Getting quotas for project ca364a2df93a424f8b66ee39d9b0b120. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 845.226535] nova-conductor[53809]: DEBUG nova.quota [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Getting quotas for user 2af2fd8431af45ca891f744f4d10b54f and project ca364a2df93a424f8b66ee39d9b0b120. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 845.234019] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: ce8c0523-8a11-4da8-af00-ee6b246ffac4] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 845.234019] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 845.234019] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 845.234019] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 845.241351] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: ce8c0523-8a11-4da8-af00-ee6b246ffac4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 845.242070] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 845.242309] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 845.242500] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 845.265938] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 845.266182] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 845.266354] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 846.124017] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell0:SELECT=24 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 846.148572] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=115,nova_cell1:SAVEPOINT=7,nova_cell1:INSERT=254,nova_cell1:RELEASE=7 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 846.156668] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=144,nova_api:DELETE=9,nova_api:UPDATE=7 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 846.161291] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=152,nova_api:UPDATE=6,nova_api:DELETE=4 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 848.121577] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: 2d8f035e-8a57-4937-a394-10d94a5630ad] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port fd9d5f30-56bf-48b3-ab8b-02cf36c2a603, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 2d8f035e-8a57-4937-a394-10d94a5630ad was re-scheduled: Binding failed for port fd9d5f30-56bf-48b3-ab8b-02cf36c2a603, please check neutron logs for more information.\n'] [ 848.121577] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 848.121577] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2d8f035e-8a57-4937-a394-10d94a5630ad.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2d8f035e-8a57-4937-a394-10d94a5630ad. [ 848.121577] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: 2d8f035e-8a57-4937-a394-10d94a5630ad] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2d8f035e-8a57-4937-a394-10d94a5630ad. [ 848.155292] nova-conductor[53810]: DEBUG nova.network.neutron [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: 2d8f035e-8a57-4937-a394-10d94a5630ad] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 848.446470] nova-conductor[53810]: DEBUG nova.network.neutron [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: 2d8f035e-8a57-4937-a394-10d94a5630ad] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 848.453604] nova-conductor[53810]: DEBUG nova.network.neutron [None req-69e21ef1-941c-4fe7-8bac-d48ac1fcda70 tempest-MultipleCreateTestJSON-759979208 tempest-MultipleCreateTestJSON-759979208-project-member] [instance: 2d8f035e-8a57-4937-a394-10d94a5630ad] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 849.017877] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] [instance: a0be018c-59be-4eb0-a3aa-b7814ce240e8] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 4f5def34-c667-41db-ab3e-33cee4e6b03a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance a0be018c-59be-4eb0-a3aa-b7814ce240e8 was re-scheduled: Binding failed for port 4f5def34-c667-41db-ab3e-33cee4e6b03a, please check neutron logs for more information.\n'] [ 849.018917] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 849.018917] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a0be018c-59be-4eb0-a3aa-b7814ce240e8.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a0be018c-59be-4eb0-a3aa-b7814ce240e8. [ 849.019176] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] [instance: a0be018c-59be-4eb0-a3aa-b7814ce240e8] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a0be018c-59be-4eb0-a3aa-b7814ce240e8. [ 849.063343] nova-conductor[53809]: DEBUG nova.network.neutron [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] [instance: a0be018c-59be-4eb0-a3aa-b7814ce240e8] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 849.152338] nova-conductor[53809]: DEBUG nova.network.neutron [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] [instance: a0be018c-59be-4eb0-a3aa-b7814ce240e8] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 849.158804] nova-conductor[53809]: DEBUG nova.network.neutron [None req-bed1e0ff-88ef-497e-bbaf-2053a82c0ba6 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] [instance: a0be018c-59be-4eb0-a3aa-b7814ce240e8] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 850.284934] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 850.304199] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 850.304199] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 850.304199] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 850.351041] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 850.351409] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 850.351499] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 850.351880] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 850.352091] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 850.352262] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 850.370697] nova-conductor[53810]: DEBUG nova.quota [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Getting quotas for project d841db9575854aa388acc0bbb499fd52. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 850.376912] nova-conductor[53810]: DEBUG nova.quota [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Getting quotas for user 721ff5c63c5f405bb7be8c486b5fd162 and project d841db9575854aa388acc0bbb499fd52. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 850.390384] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: c9574f78-b316-467b-acbe-2122886c2990] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 850.390877] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 850.391099] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 850.391269] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 850.398565] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: c9574f78-b316-467b-acbe-2122886c2990] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 850.399281] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 850.399481] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 850.399650] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 850.436218] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 850.436461] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 850.436635] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 850.451539] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 95a16efa-3240-4e15-ae19-03aaef61e2de] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 08566279-b49f-411a-97b7-79b249e6fbac, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 95a16efa-3240-4e15-ae19-03aaef61e2de was re-scheduled: Binding failed for port 08566279-b49f-411a-97b7-79b249e6fbac, please check neutron logs for more information.\n'] [ 850.452148] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 850.452404] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 95a16efa-3240-4e15-ae19-03aaef61e2de.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 95a16efa-3240-4e15-ae19-03aaef61e2de. [ 850.452692] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 95a16efa-3240-4e15-ae19-03aaef61e2de] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 95a16efa-3240-4e15-ae19-03aaef61e2de. [ 850.506670] nova-conductor[53809]: DEBUG nova.network.neutron [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 95a16efa-3240-4e15-ae19-03aaef61e2de] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 850.569419] nova-conductor[53809]: DEBUG nova.network.neutron [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 95a16efa-3240-4e15-ae19-03aaef61e2de] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 850.576043] nova-conductor[53809]: DEBUG nova.network.neutron [None req-d66a3643-738f-4c2d-a23a-245c2483597c tempest-ImagesTestJSON-1787928631 tempest-ImagesTestJSON-1787928631-project-member] [instance: 95a16efa-3240-4e15-ae19-03aaef61e2de] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 850.621614] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] [instance: 6288806c-d634-4749-8538-7188954788f0] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port da804bde-daba-479b-ab5f-0742b2a78cec, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 6288806c-d634-4749-8538-7188954788f0 was re-scheduled: Binding failed for port da804bde-daba-479b-ab5f-0742b2a78cec, please check neutron logs for more information.\n'] [ 850.623203] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 850.623590] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6288806c-d634-4749-8538-7188954788f0.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6288806c-d634-4749-8538-7188954788f0. [ 850.624044] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] [instance: 6288806c-d634-4749-8538-7188954788f0] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6288806c-d634-4749-8538-7188954788f0. [ 850.713500] nova-conductor[53810]: DEBUG nova.network.neutron [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] [instance: 6288806c-d634-4749-8538-7188954788f0] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 850.759798] nova-conductor[53810]: DEBUG nova.network.neutron [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] [instance: 6288806c-d634-4749-8538-7188954788f0] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 850.763983] nova-conductor[53810]: DEBUG nova.network.neutron [None req-bcac3c5c-f5ed-46d1-9cae-1cb4ad1bc677 tempest-ServersAdminTestJSON-620780778 tempest-ServersAdminTestJSON-620780778-project-member] [instance: 6288806c-d634-4749-8538-7188954788f0] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 855.210568] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell0:SELECT=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 855.245234] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=19 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 855.262756] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=13 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 855.263083] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=11,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 855.600211] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 855.619863] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 855.620169] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 855.620364] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 855.681358] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 855.681579] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 855.681750] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 855.682495] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 855.682495] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 855.682495] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 855.695081] nova-conductor[53809]: DEBUG nova.quota [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Getting quotas for project ec2e5213b768477f82c160e45d6ebe6b. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 855.698957] nova-conductor[53809]: DEBUG nova.quota [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Getting quotas for user 5bf57aa5f880432aac740791e76d30d8 and project ec2e5213b768477f82c160e45d6ebe6b. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 855.706241] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] [instance: d9325bb4-cb9d-4ce5-9eea-f74af6a85a2d] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 855.706619] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 855.706828] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 855.706999] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 855.711770] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] [instance: d9325bb4-cb9d-4ce5-9eea-f74af6a85a2d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 855.712562] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 855.712853] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 855.713101] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 855.734165] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 855.734275] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 855.734373] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 855.992338] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] [instance: 53400571-766c-4020-b163-87a8816199cd] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 42f49d5f-4672-4e9e-9e10-a5ab004efa63, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 53400571-766c-4020-b163-87a8816199cd was re-scheduled: Binding failed for port 42f49d5f-4672-4e9e-9e10-a5ab004efa63, please check neutron logs for more information.\n'] [ 855.992338] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 855.992338] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 53400571-766c-4020-b163-87a8816199cd.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 53400571-766c-4020-b163-87a8816199cd. [ 855.993197] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] [instance: 53400571-766c-4020-b163-87a8816199cd] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 53400571-766c-4020-b163-87a8816199cd. [ 856.022070] nova-conductor[53809]: DEBUG nova.network.neutron [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] [instance: 53400571-766c-4020-b163-87a8816199cd] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 856.077326] nova-conductor[53809]: DEBUG nova.network.neutron [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] [instance: 53400571-766c-4020-b163-87a8816199cd] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 856.080507] nova-conductor[53809]: DEBUG nova.network.neutron [None req-19035daf-7f73-468d-8208-a485a3caba8f tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] [instance: 53400571-766c-4020-b163-87a8816199cd] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 856.761507] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Took 0.22 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 856.778628] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 856.778857] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 856.779048] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 856.823601] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: ce8c0523-8a11-4da8-af00-ee6b246ffac4] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port c5a50d49-6f28-4037-a334-593256394f2c, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ce8c0523-8a11-4da8-af00-ee6b246ffac4 was re-scheduled: Binding failed for port c5a50d49-6f28-4037-a334-593256394f2c, please check neutron logs for more information.\n'] [ 856.824522] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 856.824810] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ce8c0523-8a11-4da8-af00-ee6b246ffac4.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ce8c0523-8a11-4da8-af00-ee6b246ffac4. [ 856.825043] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: ce8c0523-8a11-4da8-af00-ee6b246ffac4] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ce8c0523-8a11-4da8-af00-ee6b246ffac4. [ 856.848748] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 856.849262] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 856.849262] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 856.849877] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 856.850091] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 856.850265] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 856.855806] nova-conductor[53809]: DEBUG nova.network.neutron [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: ce8c0523-8a11-4da8-af00-ee6b246ffac4] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 856.866090] nova-conductor[53809]: DEBUG nova.quota [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Getting quotas for project de0bdfa4bde94371a7f9ec77e16e894f. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 856.869867] nova-conductor[53809]: DEBUG nova.quota [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Getting quotas for user ee5e59cd580e421c9fbed3fd04735d67 and project de0bdfa4bde94371a7f9ec77e16e894f. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 856.875790] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] [instance: eeed1f83-89d2-4887-9e44-b269a2e295ae] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 856.876286] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 856.876521] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 856.876695] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 856.879776] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] [instance: eeed1f83-89d2-4887-9e44-b269a2e295ae] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 856.880446] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 856.880666] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 856.880837] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 856.903284] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 856.903405] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 856.903840] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 856.910490] nova-conductor[53809]: DEBUG nova.network.neutron [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: ce8c0523-8a11-4da8-af00-ee6b246ffac4] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 856.914894] nova-conductor[53809]: DEBUG nova.network.neutron [None req-51a546be-8e81-49cd-94d2-ac10bfa0e01a tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: ce8c0523-8a11-4da8-af00-ee6b246ffac4] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 857.822957] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 857.840794] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 857.841081] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 857.842319] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 857.901686] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 857.901920] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 857.902099] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 857.902796] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 857.902796] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 857.902928] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 857.914980] nova-conductor[53809]: DEBUG nova.quota [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Getting quotas for project 9ff88efffe9443a39391aab1d573993a. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 857.919036] nova-conductor[53809]: DEBUG nova.quota [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Getting quotas for user 5c5974ebfc844e4c8a2947542dd55524 and project 9ff88efffe9443a39391aab1d573993a. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 857.926033] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] [instance: e51da790-9736-4181-9562-1a8f87895bd2] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 857.926505] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 857.926814] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 857.926897] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 857.933179] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] [instance: e51da790-9736-4181-9562-1a8f87895bd2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 857.933902] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 857.934189] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 857.934412] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 857.954235] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 857.954478] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 857.954657] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 860.072972] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 860.092485] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 860.092693] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 860.092873] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 860.140019] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 860.140019] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 860.140019] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 860.140019] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 860.140019] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 860.140019] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 860.150156] nova-conductor[53810]: DEBUG nova.quota [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Getting quotas for project 473b8bea722142ecbc6aa9cc36556e7a. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 860.152921] nova-conductor[53810]: DEBUG nova.quota [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Getting quotas for user 83e896d2715d4448aaed2f994e6e88e3 and project 473b8bea722142ecbc6aa9cc36556e7a. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 860.158529] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] [instance: 7f0c1eef-750c-4d8f-8d90-a02898fdeee1] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 860.159309] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 860.159560] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 860.159767] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 860.164893] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] [instance: 7f0c1eef-750c-4d8f-8d90-a02898fdeee1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 860.165587] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 860.165829] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 860.166041] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 860.181029] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 860.181029] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 860.181029] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 860.439040] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 860.460303] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 860.460303] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 860.460303] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 860.504371] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 860.505198] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 860.505305] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 860.505834] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 860.506263] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 860.506450] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 860.522789] nova-conductor[53809]: DEBUG nova.quota [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Getting quotas for project ca364a2df93a424f8b66ee39d9b0b120. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 860.529943] nova-conductor[53809]: DEBUG nova.quota [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Getting quotas for user 2af2fd8431af45ca891f744f4d10b54f and project ca364a2df93a424f8b66ee39d9b0b120. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 860.537062] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 8089bd3f-47e7-4490-8bfc-a1d87bf559ef] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 860.537699] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 860.538108] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 860.538384] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 860.548099] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 8089bd3f-47e7-4490-8bfc-a1d87bf559ef] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 860.548099] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 860.548099] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 860.548099] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 860.579875] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 860.580344] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 860.580344] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 861.032711] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: c9574f78-b316-467b-acbe-2122886c2990] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1986, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1964, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 03976688-9968-4c5b-b4cf-f1d6882dd680, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance c9574f78-b316-467b-acbe-2122886c2990 was re-scheduled: Binding failed for port 03976688-9968-4c5b-b4cf-f1d6882dd680, please check neutron logs for more information.\n'] [ 861.033361] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 861.033621] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c9574f78-b316-467b-acbe-2122886c2990.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c9574f78-b316-467b-acbe-2122886c2990. [ 861.033942] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: c9574f78-b316-467b-acbe-2122886c2990] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c9574f78-b316-467b-acbe-2122886c2990. [ 861.064478] nova-conductor[53810]: DEBUG nova.network.neutron [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: c9574f78-b316-467b-acbe-2122886c2990] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 861.117023] nova-conductor[53810]: DEBUG nova.network.neutron [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: c9574f78-b316-467b-acbe-2122886c2990] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 861.118894] nova-conductor[53810]: DEBUG nova.network.neutron [None req-fab0fa52-8b5c-420f-93a2-fcc49f6cf44e tempest-AttachVolumeNegativeTest-1225387431 tempest-AttachVolumeNegativeTest-1225387431-project-member] [instance: c9574f78-b316-467b-acbe-2122886c2990] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 865.099459] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Took 0.18 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 865.121353] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 865.121841] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 865.122187] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 865.166796] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 865.167321] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 865.167321] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 865.168824] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 865.168824] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 865.168824] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 865.179032] nova-conductor[53809]: DEBUG nova.quota [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Getting quotas for project 3bab637ad53d433db6fb2017b6c0c2aa. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 865.186920] nova-conductor[53809]: DEBUG nova.quota [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Getting quotas for user 66924fb566fb455bada0c920137fb884 and project 3bab637ad53d433db6fb2017b6c0c2aa. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 865.204960] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] [instance: b954e159-4d89-4c61-a5bc-5e6c67cf278c] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 865.206802] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 865.207179] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 865.207439] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 865.216037] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] [instance: b954e159-4d89-4c61-a5bc-5e6c67cf278c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 865.216037] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 865.216037] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 865.216037] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 865.241782] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 865.242039] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 865.242221] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 865.846392] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:UPDATE=331,nova_cell1:SELECT=1185,nova_cell1:SAVEPOINT=76,nova_cell1:RELEASE=76,nova_cell1:INSERT=52 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 867.812018] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Took 0.15 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 867.826826] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 867.827284] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 867.827676] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 867.878039] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 867.878039] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 867.878039] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 867.878039] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 867.878039] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 867.878039] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 867.886973] nova-conductor[53810]: DEBUG nova.quota [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Getting quotas for project f6b7bee2debf4cf49c138ee34e161099. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 867.890227] nova-conductor[53810]: DEBUG nova.quota [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Getting quotas for user de248172b0ac4968bbf4e4195681ea0b and project f6b7bee2debf4cf49c138ee34e161099. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 867.895938] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] [instance: 2b55d3f3-cff9-4e34-936e-ece6759cfd40] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 867.896553] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 867.896896] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 867.897231] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 867.901768] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] [instance: 2b55d3f3-cff9-4e34-936e-ece6759cfd40] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 867.902614] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 867.902948] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 867.903290] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 867.918642] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 867.919049] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 867.921071] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 868.547931] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Took 0.14 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 868.570540] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 868.570786] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 868.570954] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 868.629300] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 868.629480] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 868.629626] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 868.629996] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 868.633677] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 868.633677] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 868.647086] nova-conductor[53810]: DEBUG nova.quota [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Getting quotas for project f6b7bee2debf4cf49c138ee34e161099. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 868.651401] nova-conductor[53810]: DEBUG nova.quota [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Getting quotas for user de248172b0ac4968bbf4e4195681ea0b and project f6b7bee2debf4cf49c138ee34e161099. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 868.658217] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] [instance: aeeb7c6c-7413-46b0-8632-c7224620e9b2] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 868.658819] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 868.658882] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 868.659131] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 868.666115] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] [instance: aeeb7c6c-7413-46b0-8632-c7224620e9b2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 868.666816] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 868.666912] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 868.667142] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 868.689446] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 868.689724] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 868.689917] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 868.709241] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:UPDATE=322,nova_cell1:SELECT=1152,nova_cell1:SAVEPOINT=57,nova_cell1:RELEASE=56,nova_cell1:INSERT=51,nova_cell1:ROLLBACK=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 875.174070] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell0:SELECT=10 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 875.218628] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=45,nova_cell1:SAVEPOINT=3,nova_cell1:INSERT=98,nova_cell1:RELEASE=3 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 875.235155] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=53,nova_api:DELETE=3,nova_api:UPDATE=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 875.236455] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=67,nova_api:UPDATE=3,nova_api:DELETE=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 878.635047] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell0:SELECT=8 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 878.669629] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=36,nova_cell1:INSERT=78,nova_cell1:SAVEPOINT=2,nova_cell1:RELEASE=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 878.680729] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=48,nova_api:UPDATE=3,nova_api:DELETE=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 878.685804] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=48,nova_api:DELETE=2,nova_api:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 882.959959] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=38,nova_cell1:UPDATE=9,nova_cell1:INSERT=5,nova_cell1:SAVEPOINT=3,nova_cell1:RELEASE=3 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 884.068162] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] [instance: 42ca8a89-5938-491b-b122-deac71d18505] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 42ca8a89-5938-491b-b122-deac71d18505 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 884.068902] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 884.069175] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 42ca8a89-5938-491b-b122-deac71d18505.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 42ca8a89-5938-491b-b122-deac71d18505. [ 884.069505] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-17e35825-abc3-4ce8-be80-288f7c0e539d tempest-ServerShowV254Test-774507648 tempest-ServerShowV254Test-774507648-project-member] [instance: 42ca8a89-5938-491b-b122-deac71d18505] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 42ca8a89-5938-491b-b122-deac71d18505. [ 900.556055] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:UPDATE=25,nova_cell1:SELECT=104,nova_cell1:SAVEPOINT=5,nova_cell1:RELEASE=5 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 901.537709] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=28,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1,nova_cell1:UPDATE=4,nova_cell1:INSERT=2,nova_cell1:DELETE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 934.150984] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] [instance: bf9101c9-4072-4f72-8ac3-24b7a5b88b45] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance bf9101c9-4072-4f72-8ac3-24b7a5b88b45 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 934.151372] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 934.151571] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance bf9101c9-4072-4f72-8ac3-24b7a5b88b45.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance bf9101c9-4072-4f72-8ac3-24b7a5b88b45. [ 934.151801] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-f39ff465-836e-48cf-9c6e-b3ad3fbeaba5 tempest-ServerDiagnosticsV248Test-537127248 tempest-ServerDiagnosticsV248Test-537127248-project-member] [instance: bf9101c9-4072-4f72-8ac3-24b7a5b88b45] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance bf9101c9-4072-4f72-8ac3-24b7a5b88b45. [ 937.237685] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 937.250135] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 937.250392] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 937.250595] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 937.285676] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 937.285676] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 937.286583] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 937.286583] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 937.286583] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 937.286583] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 937.294244] nova-conductor[53809]: DEBUG nova.quota [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Getting quotas for project b72c7e53e97e46efa6a0fd8ccf211f1d. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 937.296541] nova-conductor[53809]: DEBUG nova.quota [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Getting quotas for user 0d672fb4dc3048a4a1bc300f0d91f1b7 and project b72c7e53e97e46efa6a0fd8ccf211f1d. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 937.302403] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] [instance: 7b48729f-86a7-4b53-ad11-ef8a929ec947] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 937.303332] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 937.303332] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 937.303332] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 937.307825] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] [instance: 7b48729f-86a7-4b53-ad11-ef8a929ec947] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 937.308461] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 937.308665] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 937.308837] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 937.323934] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 937.324161] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 937.324335] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 947.290061] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell0:SELECT=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 947.311830] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=9,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=20,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 947.317915] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=11 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 947.320891] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 951.132842] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=46,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1,nova_cell1:UPDATE=10,nova_cell1:INSERT=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 964.204457] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=8,nova_cell1:UPDATE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 968.256601] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=59,nova_cell1:UPDATE=16,nova_cell1:SAVEPOINT=3,nova_cell1:RELEASE=3,nova_cell1:INSERT=3 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 984.337010] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] [instance: eeed1f83-89d2-4887-9e44-b269a2e295ae] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance eeed1f83-89d2-4887-9e44-b269a2e295ae was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 984.337602] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 984.337850] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance eeed1f83-89d2-4887-9e44-b269a2e295ae.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance eeed1f83-89d2-4887-9e44-b269a2e295ae. [ 984.338214] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-f7fdaaad-da75-4239-ade5-65b6d66b06b5 tempest-ServersAaction247Test-2099232595 tempest-ServersAaction247Test-2099232595-project-member] [instance: eeed1f83-89d2-4887-9e44-b269a2e295ae] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance eeed1f83-89d2-4887-9e44-b269a2e295ae. [ 1002.963946] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=23,nova_cell1:UPDATE=8,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1002.964535] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=20,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1,nova_cell1:UPDATE=5,nova_cell1:INSERT=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1021.527898] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=6,nova_cell1:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1024.209162] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=7 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1034.721966] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] [instance: d9325bb4-cb9d-4ce5-9eea-f74af6a85a2d] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance d9325bb4-cb9d-4ce5-9eea-f74af6a85a2d was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1034.722446] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1034.722694] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d9325bb4-cb9d-4ce5-9eea-f74af6a85a2d.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d9325bb4-cb9d-4ce5-9eea-f74af6a85a2d. [ 1034.722997] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] [instance: d9325bb4-cb9d-4ce5-9eea-f74af6a85a2d] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d9325bb4-cb9d-4ce5-9eea-f74af6a85a2d. [ 1034.747753] nova-conductor[53809]: DEBUG nova.network.neutron [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] [instance: d9325bb4-cb9d-4ce5-9eea-f74af6a85a2d] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1034.832878] nova-conductor[53809]: DEBUG nova.network.neutron [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] [instance: d9325bb4-cb9d-4ce5-9eea-f74af6a85a2d] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1034.837398] nova-conductor[53809]: DEBUG nova.network.neutron [None req-0a63ee0a-42b4-4edf-a643-bd214151c0e5 tempest-ServerTagsTestJSON-568839792 tempest-ServerTagsTestJSON-568839792-project-member] [instance: d9325bb4-cb9d-4ce5-9eea-f74af6a85a2d] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1044.731520] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=19,nova_cell1:UPDATE=8,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1044.847481] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=18,nova_cell1:UPDATE=7,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1071.645389] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=4,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1081.123483] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] [instance: e51da790-9736-4181-9562-1a8f87895bd2] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance e51da790-9736-4181-9562-1a8f87895bd2 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1081.124118] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1081.124388] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e51da790-9736-4181-9562-1a8f87895bd2.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e51da790-9736-4181-9562-1a8f87895bd2. [ 1081.124616] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] [instance: e51da790-9736-4181-9562-1a8f87895bd2] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e51da790-9736-4181-9562-1a8f87895bd2. [ 1081.202550] nova-conductor[53809]: DEBUG nova.network.neutron [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] [instance: e51da790-9736-4181-9562-1a8f87895bd2] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1081.222441] nova-conductor[53809]: DEBUG nova.network.neutron [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] [instance: e51da790-9736-4181-9562-1a8f87895bd2] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1081.225915] nova-conductor[53809]: DEBUG nova.network.neutron [None req-e6f19eca-71a3-4525-8921-22c7ebc68175 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] [instance: e51da790-9736-4181-9562-1a8f87895bd2] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1084.564030] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1084.586629] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1084.586862] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1084.587047] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1084.629128] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1084.629363] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1084.629537] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1084.629912] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1084.630113] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1084.630280] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1084.638611] nova-conductor[53810]: DEBUG nova.quota [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Getting quotas for project 9ff88efffe9443a39391aab1d573993a. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1084.641447] nova-conductor[53810]: DEBUG nova.quota [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Getting quotas for user 5c5974ebfc844e4c8a2947542dd55524 and project 9ff88efffe9443a39391aab1d573993a. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1084.647570] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] [instance: 42999bc8-a3be-4310-97ad-324c7f4fc8d4] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1084.647921] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1084.648145] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1084.648362] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1084.678375] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] [instance: 42999bc8-a3be-4310-97ad-324c7f4fc8d4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1084.679077] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1084.679296] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1084.679481] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1084.709309] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1084.709563] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1084.709717] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1094.634869] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell0:SELECT=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1094.681981] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=19 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1094.700114] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=11,nova_api:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1094.704685] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=13,nova_api:DELETE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1098.561925] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=61,nova_cell1:UPDATE=26,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=2,nova_cell1:RELEASE=2,nova_cell1:DELETE=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1098.840641] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=70,nova_cell1:UPDATE=24,nova_cell1:INSERT=7,nova_cell1:SAVEPOINT=3,nova_cell1:RELEASE=3 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1122.961873] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=2,nova_cell1:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1122.969404] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1128.469027] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 8089bd3f-47e7-4490-8bfc-a1d87bf559ef] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 8089bd3f-47e7-4490-8bfc-a1d87bf559ef was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1128.469392] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1128.469548] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8089bd3f-47e7-4490-8bfc-a1d87bf559ef.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8089bd3f-47e7-4490-8bfc-a1d87bf559ef. [ 1128.469812] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 8089bd3f-47e7-4490-8bfc-a1d87bf559ef] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8089bd3f-47e7-4490-8bfc-a1d87bf559ef. [ 1128.493427] nova-conductor[53809]: DEBUG nova.network.neutron [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 8089bd3f-47e7-4490-8bfc-a1d87bf559ef] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1128.512328] nova-conductor[53809]: DEBUG nova.network.neutron [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 8089bd3f-47e7-4490-8bfc-a1d87bf559ef] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1128.515344] nova-conductor[53809]: DEBUG nova.network.neutron [None req-27ca987c-126d-47a1-8cf5-daadd80d35a6 tempest-DeleteServersTestJSON-64622698 tempest-DeleteServersTestJSON-64622698-project-member] [instance: 8089bd3f-47e7-4490-8bfc-a1d87bf559ef] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1134.322656] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Took 0.14 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1134.336337] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1134.339488] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1134.339488] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1134.391119] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1134.392024] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1134.392024] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1134.392024] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1134.392024] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1134.392277] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1134.401242] nova-conductor[53809]: DEBUG nova.quota [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Getting quotas for project 2657ae3507d340b6950c02b104d16f7d. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1134.404017] nova-conductor[53809]: DEBUG nova.quota [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Getting quotas for user 576b47a5c78d458fa934561690148b4e and project 2657ae3507d340b6950c02b104d16f7d. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1134.410181] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] [instance: 5403acfb-04d2-4081-80c7-23d662410a20] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1134.410767] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1134.411032] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1134.411253] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1134.415951] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] [instance: 5403acfb-04d2-4081-80c7-23d662410a20] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1134.416712] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1134.416925] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1134.417348] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1134.430422] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1134.430752] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1134.430968] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1144.396872] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell0:SELECT=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1144.419936] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=9,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=20,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1144.424117] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=9 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1144.430112] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=15,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1147.930104] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=60,nova_cell1:UPDATE=27,nova_cell1:INSERT=5,nova_cell1:SAVEPOINT=2,nova_cell1:RELEASE=2,nova_cell1:DELETE=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1148.160640] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=65,nova_cell1:UPDATE=20,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=3,nova_cell1:RELEASE=3 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1179.612924] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] [instance: 7f0c1eef-750c-4d8f-8d90-a02898fdeee1] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 7f0c1eef-750c-4d8f-8d90-a02898fdeee1 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1179.613526] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1179.613809] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7f0c1eef-750c-4d8f-8d90-a02898fdeee1.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7f0c1eef-750c-4d8f-8d90-a02898fdeee1. [ 1179.614156] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] [instance: 7f0c1eef-750c-4d8f-8d90-a02898fdeee1] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7f0c1eef-750c-4d8f-8d90-a02898fdeee1. [ 1179.657440] nova-conductor[53809]: DEBUG nova.network.neutron [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] [instance: 7f0c1eef-750c-4d8f-8d90-a02898fdeee1] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1179.675027] nova-conductor[53809]: DEBUG nova.network.neutron [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] [instance: 7f0c1eef-750c-4d8f-8d90-a02898fdeee1] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1179.677959] nova-conductor[53809]: DEBUG nova.network.neutron [None req-da2c9d5f-617a-4658-a4f7-9a2b8b18d74e tempest-ServerRescueTestJSON-1180270528 tempest-ServerRescueTestJSON-1180270528-project-member] [instance: 7f0c1eef-750c-4d8f-8d90-a02898fdeee1] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1189.777045] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=25,nova_cell1:UPDATE=19,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=2,nova_cell1:RELEASE=2,nova_cell1:DELETE=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1189.792798] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=12,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1210.213938] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=12,nova_cell1:DELETE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1214.251174] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=12,nova_cell1:UPDATE=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1230.574690] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] [instance: 2b55d3f3-cff9-4e34-936e-ece6759cfd40] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 2b55d3f3-cff9-4e34-936e-ece6759cfd40 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1230.575283] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1230.575513] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2b55d3f3-cff9-4e34-936e-ece6759cfd40.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2b55d3f3-cff9-4e34-936e-ece6759cfd40. [ 1230.575760] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] [instance: 2b55d3f3-cff9-4e34-936e-ece6759cfd40] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2b55d3f3-cff9-4e34-936e-ece6759cfd40. [ 1230.595921] nova-conductor[53809]: DEBUG nova.network.neutron [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] [instance: 2b55d3f3-cff9-4e34-936e-ece6759cfd40] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1230.615136] nova-conductor[53809]: DEBUG nova.network.neutron [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] [instance: 2b55d3f3-cff9-4e34-936e-ece6759cfd40] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1230.617918] nova-conductor[53809]: DEBUG nova.network.neutron [None req-325872bc-0f93-4c55-a545-7c70ed051360 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] [instance: 2b55d3f3-cff9-4e34-936e-ece6759cfd40] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1242.956333] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=21,nova_cell1:UPDATE=9,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1242.973471] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=20,nova_cell1:UPDATE=7,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1264.148386] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Took 0.27 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1264.168865] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1264.169114] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1264.169292] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1264.204617] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1264.204894] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1264.205086] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1264.205509] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1264.205659] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1264.205821] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1264.217787] nova-conductor[53809]: DEBUG nova.quota [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Getting quotas for project e839303682f748a4b5a42c8a9273e388. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1264.220152] nova-conductor[53809]: DEBUG nova.quota [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Getting quotas for user cf4545757716483485ca9b60bd689a1c and project e839303682f748a4b5a42c8a9273e388. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1264.226442] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: ba1bd6f7-6f05-4a09-a35c-6493b64feb9d] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1264.226966] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1264.227209] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1264.227384] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1264.232504] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: ba1bd6f7-6f05-4a09-a35c-6493b64feb9d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1264.233163] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1264.233371] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1264.233585] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1264.250084] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1264.250084] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1264.250084] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1265.486829] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1265.504684] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1265.504684] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1265.504684] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1265.547406] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1265.547665] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1265.547858] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1265.548239] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1265.548430] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1265.548595] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1265.558227] nova-conductor[53810]: DEBUG nova.quota [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Getting quotas for project 32ed96e3b417472da91886ea192c588b. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1265.560778] nova-conductor[53810]: DEBUG nova.quota [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Getting quotas for user 05fe4841eb644cc792f699e538d31b7f and project 32ed96e3b417472da91886ea192c588b. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1265.566456] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] [instance: 5fb1f6f5-7c20-4b89-ae2f-0ab2a416b53a] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1265.566936] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1265.567257] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1265.567437] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1265.572561] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] [instance: 5fb1f6f5-7c20-4b89-ae2f-0ab2a416b53a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1265.573465] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1265.573612] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1265.573704] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1265.588892] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1265.589118] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1265.589301] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1266.052021] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Took 0.14 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1266.070825] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1266.071344] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1266.072011] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1266.103946] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1266.104203] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1266.104374] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1266.104739] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1266.108015] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1266.108015] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1266.114825] nova-conductor[53809]: DEBUG nova.quota [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Getting quotas for project 32ed96e3b417472da91886ea192c588b. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1266.118208] nova-conductor[53809]: DEBUG nova.quota [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Getting quotas for user 05fe4841eb644cc792f699e538d31b7f and project 32ed96e3b417472da91886ea192c588b. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1266.125081] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] [instance: 4abe5722-e83b-4c40-9b82-ca84545496c8] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1266.125438] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1266.125583] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1266.125990] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1266.132089] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] [instance: 4abe5722-e83b-4c40-9b82-ca84545496c8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1266.132799] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1266.133012] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1266.133187] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1266.149593] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1266.149878] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1266.149987] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1275.554220] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell0:SELECT=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1275.576238] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=9,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=20,nova_cell1:RELEASE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1275.586396] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=11,nova_api:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1275.587082] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=13,nova_api:DELETE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1276.110196] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell0:SELECT=4 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1276.135339] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=18,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=39,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1276.144508] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=19 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1276.147430] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=29,nova_api:UPDATE=2,nova_api:DELETE=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1279.648227] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] [instance: b954e159-4d89-4c61-a5bc-5e6c67cf278c] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance b954e159-4d89-4c61-a5bc-5e6c67cf278c was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1279.648828] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1279.649078] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b954e159-4d89-4c61-a5bc-5e6c67cf278c.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b954e159-4d89-4c61-a5bc-5e6c67cf278c. [ 1279.649466] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] [instance: b954e159-4d89-4c61-a5bc-5e6c67cf278c] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b954e159-4d89-4c61-a5bc-5e6c67cf278c. [ 1279.675475] nova-conductor[53810]: DEBUG nova.network.neutron [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] [instance: b954e159-4d89-4c61-a5bc-5e6c67cf278c] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1279.729286] nova-conductor[53810]: DEBUG nova.network.neutron [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] [instance: b954e159-4d89-4c61-a5bc-5e6c67cf278c] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1279.732466] nova-conductor[53810]: DEBUG nova.network.neutron [None req-cf626a5a-7b41-4975-8a7e-32f61e1bb58b tempest-ServersTestMultiNic-72568094 tempest-ServersTestMultiNic-72568094-project-member] [instance: b954e159-4d89-4c61-a5bc-5e6c67cf278c] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1289.840045] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=121,nova_cell1:UPDATE=34,nova_cell1:INSERT=7,nova_cell1:SAVEPOINT=5,nova_cell1:RELEASE=5 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1290.477607] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=112,nova_cell1:UPDATE=41,nova_cell1:INSERT=5,nova_cell1:SAVEPOINT=4,nova_cell1:RELEASE=4,nova_cell1:DELETE=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1329.696530] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] [instance: aeeb7c6c-7413-46b0-8632-c7224620e9b2] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance aeeb7c6c-7413-46b0-8632-c7224620e9b2 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1329.696530] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1329.696530] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance aeeb7c6c-7413-46b0-8632-c7224620e9b2.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance aeeb7c6c-7413-46b0-8632-c7224620e9b2. [ 1329.696530] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] [instance: aeeb7c6c-7413-46b0-8632-c7224620e9b2] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance aeeb7c6c-7413-46b0-8632-c7224620e9b2. [ 1329.729781] nova-conductor[53809]: DEBUG nova.network.neutron [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] [instance: aeeb7c6c-7413-46b0-8632-c7224620e9b2] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1329.748651] nova-conductor[53809]: DEBUG nova.network.neutron [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] [instance: aeeb7c6c-7413-46b0-8632-c7224620e9b2] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1329.753035] nova-conductor[53809]: DEBUG nova.network.neutron [None req-8d2d9ebc-cb54-427f-bf54-f83a78f24582 tempest-ListImageFiltersTestJSON-1130217809 tempest-ListImageFiltersTestJSON-1130217809-project-member] [instance: aeeb7c6c-7413-46b0-8632-c7224620e9b2] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1339.884784] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=37,nova_cell1:UPDATE=13,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1340.275182] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=35,nova_cell1:UPDATE=21,nova_cell1:INSERT=2,nova_cell1:SAVEPOINT=2,nova_cell1:RELEASE=2,nova_cell1:DELETE=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1362.954268] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=2,nova_cell1:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1362.978373] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1380.657447] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] [instance: 7b48729f-86a7-4b53-ad11-ef8a929ec947] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 7b48729f-86a7-4b53-ad11-ef8a929ec947 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1380.657447] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1380.657447] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7b48729f-86a7-4b53-ad11-ef8a929ec947.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7b48729f-86a7-4b53-ad11-ef8a929ec947. [ 1380.657447] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] [instance: 7b48729f-86a7-4b53-ad11-ef8a929ec947] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7b48729f-86a7-4b53-ad11-ef8a929ec947. [ 1380.690589] nova-conductor[53809]: DEBUG nova.network.neutron [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] [instance: 7b48729f-86a7-4b53-ad11-ef8a929ec947] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1380.774161] nova-conductor[53809]: DEBUG nova.network.neutron [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] [instance: 7b48729f-86a7-4b53-ad11-ef8a929ec947] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1380.779630] nova-conductor[53809]: DEBUG nova.network.neutron [None req-fe6b4a7f-112c-4b01-bf92-4abb4412760d tempest-ServerDiagnosticsNegativeTest-937147690 tempest-ServerDiagnosticsNegativeTest-937147690-project-member] [instance: 7b48729f-86a7-4b53-ad11-ef8a929ec947] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1390.848661] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=34,nova_cell1:UPDATE=13,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1392.209989] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=32,nova_cell1:UPDATE=19,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=2,nova_cell1:RELEASE=2,nova_cell1:DELETE=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1429.002623] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] [instance: 42999bc8-a3be-4310-97ad-324c7f4fc8d4] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 42999bc8-a3be-4310-97ad-324c7f4fc8d4 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1429.003278] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1429.003491] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 42999bc8-a3be-4310-97ad-324c7f4fc8d4.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 42999bc8-a3be-4310-97ad-324c7f4fc8d4. [ 1429.003773] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] [instance: 42999bc8-a3be-4310-97ad-324c7f4fc8d4] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 42999bc8-a3be-4310-97ad-324c7f4fc8d4. [ 1429.028439] nova-conductor[53810]: DEBUG nova.network.neutron [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] [instance: 42999bc8-a3be-4310-97ad-324c7f4fc8d4] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1429.045996] nova-conductor[53810]: DEBUG nova.network.neutron [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] [instance: 42999bc8-a3be-4310-97ad-324c7f4fc8d4] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1429.049010] nova-conductor[53810]: DEBUG nova.network.neutron [None req-11275d1e-ec11-4e51-998a-8c3434f8bc46 tempest-AttachVolumeShelveTestJSON-664442013 tempest-AttachVolumeShelveTestJSON-664442013-project-member] [instance: 42999bc8-a3be-4310-97ad-324c7f4fc8d4] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1439.175152] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=12,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1453.203751] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=34,nova_cell1:UPDATE=20,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=2,nova_cell1:RELEASE=2,nova_cell1:DELETE=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1453.218481] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=8,nova_cell1:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1469.581060] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=2,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1471.731656] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=2,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1479.340036] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] [instance: 5403acfb-04d2-4081-80c7-23d662410a20] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 5403acfb-04d2-4081-80c7-23d662410a20 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1479.340745] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1479.340745] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5403acfb-04d2-4081-80c7-23d662410a20.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5403acfb-04d2-4081-80c7-23d662410a20. [ 1479.340959] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] [instance: 5403acfb-04d2-4081-80c7-23d662410a20] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5403acfb-04d2-4081-80c7-23d662410a20. [ 1479.365530] nova-conductor[53810]: DEBUG nova.network.neutron [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] [instance: 5403acfb-04d2-4081-80c7-23d662410a20] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1479.383046] nova-conductor[53810]: DEBUG nova.network.neutron [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] [instance: 5403acfb-04d2-4081-80c7-23d662410a20] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1479.386315] nova-conductor[53810]: DEBUG nova.network.neutron [None req-ce69b387-7179-4a79-8a3e-67ee1f5a2407 tempest-InstanceActionsNegativeTestJSON-667127095 tempest-InstanceActionsNegativeTestJSON-667127095-project-member] [instance: 5403acfb-04d2-4081-80c7-23d662410a20] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1489.492547] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=27,nova_cell1:UPDATE=19,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=2,nova_cell1:RELEASE=2,nova_cell1:DELETE=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1489.509526] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=30,nova_cell1:UPDATE=13,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1514.208496] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=8,nova_cell1:DELETE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1514.223070] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=10,nova_cell1:UPDATE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1529.271028] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: ba1bd6f7-6f05-4a09-a35c-6493b64feb9d] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance ba1bd6f7-6f05-4a09-a35c-6493b64feb9d was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1529.271748] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1529.271874] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ba1bd6f7-6f05-4a09-a35c-6493b64feb9d.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ba1bd6f7-6f05-4a09-a35c-6493b64feb9d. [ 1529.272092] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: ba1bd6f7-6f05-4a09-a35c-6493b64feb9d] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ba1bd6f7-6f05-4a09-a35c-6493b64feb9d. [ 1529.292271] nova-conductor[53809]: DEBUG nova.network.neutron [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: ba1bd6f7-6f05-4a09-a35c-6493b64feb9d] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1529.310164] nova-conductor[53809]: DEBUG nova.network.neutron [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: ba1bd6f7-6f05-4a09-a35c-6493b64feb9d] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1529.313367] nova-conductor[53809]: DEBUG nova.network.neutron [None req-0440fba7-33d5-4895-8b0f-4c0c424b327d tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: ba1bd6f7-6f05-4a09-a35c-6493b64feb9d] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1530.081152] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-cbcae74b-6d78-46e2-8a0e-d08a1e3e16bc tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Took 0.12 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1530.089481] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-cbcae74b-6d78-46e2-8a0e-d08a1e3e16bc tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: c38705da-781f-4399-b415-1dab38fb9c06] While scheduling instance, the build request was already deleted. {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1706}} [ 1530.140900] nova-conductor[53810]: INFO nova.scheduler.client.report [None req-cbcae74b-6d78-46e2-8a0e-d08a1e3e16bc tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Deleted allocations for instance c38705da-781f-4399-b415-1dab38fb9c06 [ 1530.145996] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-cbcae74b-6d78-46e2-8a0e-d08a1e3e16bc tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1530.146245] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-cbcae74b-6d78-46e2-8a0e-d08a1e3e16bc tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1530.146422] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-cbcae74b-6d78-46e2-8a0e-d08a1e3e16bc tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1530.146778] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-cbcae74b-6d78-46e2-8a0e-d08a1e3e16bc tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1530.146970] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-cbcae74b-6d78-46e2-8a0e-d08a1e3e16bc tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1530.147192] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-cbcae74b-6d78-46e2-8a0e-d08a1e3e16bc tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1530.156542] nova-conductor[53810]: DEBUG nova.quota [None req-cbcae74b-6d78-46e2-8a0e-d08a1e3e16bc tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Getting quotas for project e839303682f748a4b5a42c8a9273e388. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1530.158788] nova-conductor[53810]: DEBUG nova.quota [None req-cbcae74b-6d78-46e2-8a0e-d08a1e3e16bc tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Getting quotas for user cf4545757716483485ca9b60bd689a1c and project e839303682f748a4b5a42c8a9273e388. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1530.541143] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Took 0.11 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1530.552850] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1530.553093] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1530.553270] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1530.590266] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1530.590489] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1530.590663] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1530.591050] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1530.591240] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1530.591401] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1530.599443] nova-conductor[53809]: DEBUG nova.quota [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Getting quotas for project e839303682f748a4b5a42c8a9273e388. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1530.601641] nova-conductor[53809]: DEBUG nova.quota [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Getting quotas for user cf4545757716483485ca9b60bd689a1c and project e839303682f748a4b5a42c8a9273e388. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1530.606979] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 7c22930b-a1b7-46b7-82ae-de363f85b393] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1530.607418] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1530.607623] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1530.607793] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1530.612699] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 7c22930b-a1b7-46b7-82ae-de363f85b393] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1530.613332] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1530.613533] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1530.613700] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1530.629319] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1530.629379] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1530.629531] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1540.151279] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell0:SELECT=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1540.152454] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1540.160817] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=9,nova_api:DELETE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1540.163476] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=10 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1540.595075] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell0:SELECT=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1540.616180] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=19 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1540.623125] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=8,nova_api:UPDATE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1540.625673] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=16,nova_api:DELETE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1543.987313] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=56,nova_cell1:UPDATE=18,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1544.304897] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=58,nova_cell1:UPDATE=27,nova_cell1:INSERT=4,nova_cell1:SAVEPOINT=4,nova_cell1:RELEASE=4,nova_cell1:DELETE=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1575.208378] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=9,nova_cell1:UPDATE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1577.217754] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=8,nova_cell1:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1579.539036] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] [instance: 5fb1f6f5-7c20-4b89-ae2f-0ab2a416b53a] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 5fb1f6f5-7c20-4b89-ae2f-0ab2a416b53a was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1579.539416] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1579.539637] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5fb1f6f5-7c20-4b89-ae2f-0ab2a416b53a.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5fb1f6f5-7c20-4b89-ae2f-0ab2a416b53a. [ 1579.539882] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] [instance: 5fb1f6f5-7c20-4b89-ae2f-0ab2a416b53a] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5fb1f6f5-7c20-4b89-ae2f-0ab2a416b53a. [ 1579.569622] nova-conductor[53810]: DEBUG nova.network.neutron [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] [instance: 5fb1f6f5-7c20-4b89-ae2f-0ab2a416b53a] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1579.612036] nova-conductor[53810]: DEBUG nova.network.neutron [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] [instance: 5fb1f6f5-7c20-4b89-ae2f-0ab2a416b53a] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1579.615255] nova-conductor[53810]: DEBUG nova.network.neutron [None req-96438c0a-d283-4012-8477-59279c736c1f tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] [instance: 5fb1f6f5-7c20-4b89-ae2f-0ab2a416b53a] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1589.547410] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=19,nova_cell1:UPDATE=8,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1589.625433] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=18,nova_cell1:UPDATE=7,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1602.964254] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=2,nova_cell1:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1602.986915] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1629.760523] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] [instance: 4abe5722-e83b-4c40-9b82-ca84545496c8] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 4abe5722-e83b-4c40-9b82-ca84545496c8 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1629.761447] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1629.761915] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4abe5722-e83b-4c40-9b82-ca84545496c8.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4abe5722-e83b-4c40-9b82-ca84545496c8. [ 1629.762869] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] [instance: 4abe5722-e83b-4c40-9b82-ca84545496c8] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4abe5722-e83b-4c40-9b82-ca84545496c8. [ 1629.785586] nova-conductor[53809]: DEBUG nova.network.neutron [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] [instance: 4abe5722-e83b-4c40-9b82-ca84545496c8] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1629.804640] nova-conductor[53809]: DEBUG nova.network.neutron [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] [instance: 4abe5722-e83b-4c40-9b82-ca84545496c8] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1629.807891] nova-conductor[53809]: DEBUG nova.network.neutron [None req-eec01ff8-31b7-4d0a-a7d3-549c6cf24ebf tempest-ServerRescueNegativeTestJSON-1975896800 tempest-ServerRescueNegativeTestJSON-1975896800-project-member] [instance: 4abe5722-e83b-4c40-9b82-ca84545496c8] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1639.931991] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=31,nova_cell1:UPDATE=20,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=2,nova_cell1:RELEASE=2,nova_cell1:DELETE=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1639.949619] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=35,nova_cell1:UPDATE=12,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1680.579113] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 7c22930b-a1b7-46b7-82ae-de363f85b393] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 7c22930b-a1b7-46b7-82ae-de363f85b393 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1680.579695] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1680.579930] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7c22930b-a1b7-46b7-82ae-de363f85b393.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7c22930b-a1b7-46b7-82ae-de363f85b393. [ 1680.580174] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 7c22930b-a1b7-46b7-82ae-de363f85b393] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7c22930b-a1b7-46b7-82ae-de363f85b393. [ 1680.608814] nova-conductor[53810]: DEBUG nova.network.neutron [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 7c22930b-a1b7-46b7-82ae-de363f85b393] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1680.626306] nova-conductor[53810]: DEBUG nova.network.neutron [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 7c22930b-a1b7-46b7-82ae-de363f85b393] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1680.629118] nova-conductor[53810]: DEBUG nova.network.neutron [None req-8393179c-baf6-49ae-bb58-2638f5142f9a tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 7c22930b-a1b7-46b7-82ae-de363f85b393] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1682.154015] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Took 0.18 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1682.167523] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1682.167749] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1682.167926] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1682.201236] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1682.201453] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1682.201625] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1682.201984] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1682.202192] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1682.202356] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1682.211223] nova-conductor[53810]: DEBUG nova.quota [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Getting quotas for project e839303682f748a4b5a42c8a9273e388. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1682.214371] nova-conductor[53810]: DEBUG nova.quota [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Getting quotas for user cf4545757716483485ca9b60bd689a1c and project e839303682f748a4b5a42c8a9273e388. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1682.219703] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 77bcc713-fa13-4c98-97b7-7ea1697d0426] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1682.220127] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1682.220331] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1682.220502] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1682.224958] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 77bcc713-fa13-4c98-97b7-7ea1697d0426] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1682.225584] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1682.225788] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1682.225960] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1682.242012] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1682.242233] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1682.242406] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1692.206156] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell0:SELECT=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1692.228057] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=19 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1692.239018] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=10,nova_api:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1692.240366] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=14,nova_api:DELETE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1697.208313] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=53,nova_cell1:UPDATE=16,nova_cell1:INSERT=3,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1702.205104] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=56,nova_cell1:UPDATE=15,nova_cell1:INSERT=2,nova_cell1:SAVEPOINT=3,nova_cell1:RELEASE=3 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1722.961419] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=2,nova_cell1:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1722.992120] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1734.692089] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 77bcc713-fa13-4c98-97b7-7ea1697d0426] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 77bcc713-fa13-4c98-97b7-7ea1697d0426 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1734.692716] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1734.693050] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 77bcc713-fa13-4c98-97b7-7ea1697d0426.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 77bcc713-fa13-4c98-97b7-7ea1697d0426. [ 1734.693383] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 77bcc713-fa13-4c98-97b7-7ea1697d0426] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 77bcc713-fa13-4c98-97b7-7ea1697d0426. [ 1734.713704] nova-conductor[53810]: DEBUG nova.network.neutron [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 77bcc713-fa13-4c98-97b7-7ea1697d0426] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1734.731105] nova-conductor[53810]: DEBUG nova.network.neutron [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 77bcc713-fa13-4c98-97b7-7ea1697d0426] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1734.734305] nova-conductor[53810]: DEBUG nova.network.neutron [None req-36f76f82-7da8-4380-bbd5-128f92b9eb8c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 77bcc713-fa13-4c98-97b7-7ea1697d0426] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1735.951249] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Took 0.12 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1735.964403] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1735.964590] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1735.964765] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1735.994758] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1735.994967] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1735.995154] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1735.995517] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1735.995704] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1735.995872] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1736.003483] nova-conductor[53810]: DEBUG nova.quota [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Getting quotas for project e839303682f748a4b5a42c8a9273e388. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1736.005767] nova-conductor[53810]: DEBUG nova.quota [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Getting quotas for user cf4545757716483485ca9b60bd689a1c and project e839303682f748a4b5a42c8a9273e388. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1736.011117] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 53c6f49c-42cc-49c3-a3b4-517a96f0502c] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1736.011509] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1736.011708] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1736.011876] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1736.015892] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 53c6f49c-42cc-49c3-a3b4-517a96f0502c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1736.016503] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1736.016704] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1736.016872] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1736.031229] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1736.031416] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1736.031589] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1746.000096] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell0:SELECT=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1746.019042] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=19 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1746.026919] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=10 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1746.029824] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=14,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1754.413461] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=52,nova_cell1:UPDATE=14,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1757.208273] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=53,nova_cell1:UPDATE=16,nova_cell1:INSERT=4,nova_cell1:SAVEPOINT=3,nova_cell1:RELEASE=3 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1789.690164] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 53c6f49c-42cc-49c3-a3b4-517a96f0502c] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 53c6f49c-42cc-49c3-a3b4-517a96f0502c was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1789.690740] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1789.690992] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 53c6f49c-42cc-49c3-a3b4-517a96f0502c.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 53c6f49c-42cc-49c3-a3b4-517a96f0502c. [ 1789.691227] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 53c6f49c-42cc-49c3-a3b4-517a96f0502c] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 53c6f49c-42cc-49c3-a3b4-517a96f0502c. [ 1789.714375] nova-conductor[53809]: DEBUG nova.network.neutron [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 53c6f49c-42cc-49c3-a3b4-517a96f0502c] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1789.782273] nova-conductor[53809]: DEBUG nova.network.neutron [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 53c6f49c-42cc-49c3-a3b4-517a96f0502c] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1789.786119] nova-conductor[53809]: DEBUG nova.network.neutron [None req-82e51024-9f2d-4582-a8a9-9b00c143581c tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 53c6f49c-42cc-49c3-a3b4-517a96f0502c] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1790.697524] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Took 0.12 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1790.710334] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1790.710581] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1790.710759] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1790.738805] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1790.739028] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1790.739209] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1790.739589] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1790.739845] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1790.740057] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1790.747802] nova-conductor[53809]: DEBUG nova.quota [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Getting quotas for project e839303682f748a4b5a42c8a9273e388. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1790.750041] nova-conductor[53809]: DEBUG nova.quota [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Getting quotas for user cf4545757716483485ca9b60bd689a1c and project e839303682f748a4b5a42c8a9273e388. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1790.755691] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: d25ad8a1-1f52-4e6b-93a5-629420a06e66] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1790.756074] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1790.756278] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1790.756446] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1790.762617] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: d25ad8a1-1f52-4e6b-93a5-629420a06e66] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1790.763265] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1790.763468] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1790.763646] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1790.776963] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1790.777192] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1790.777366] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1800.744327] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell0:SELECT=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1800.766150] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=19 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1800.773016] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=11 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1800.775942] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1817.220053] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=56,nova_cell1:UPDATE=18,nova_cell1:INSERT=4,nova_cell1:SAVEPOINT=3,nova_cell1:RELEASE=3 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1824.204764] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=58,nova_cell1:UPDATE=13,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1,nova_cell1:DELETE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1842.957176] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=4,nova_cell1:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1842.995872] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1844.559743] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: d25ad8a1-1f52-4e6b-93a5-629420a06e66] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance d25ad8a1-1f52-4e6b-93a5-629420a06e66 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1844.560566] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1844.560713] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d25ad8a1-1f52-4e6b-93a5-629420a06e66.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d25ad8a1-1f52-4e6b-93a5-629420a06e66. [ 1844.561059] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: d25ad8a1-1f52-4e6b-93a5-629420a06e66] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d25ad8a1-1f52-4e6b-93a5-629420a06e66. [ 1844.583250] nova-conductor[53810]: DEBUG nova.network.neutron [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: d25ad8a1-1f52-4e6b-93a5-629420a06e66] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1844.600802] nova-conductor[53810]: DEBUG nova.network.neutron [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: d25ad8a1-1f52-4e6b-93a5-629420a06e66] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1844.603664] nova-conductor[53810]: DEBUG nova.network.neutron [None req-4e09a1e0-3fc1-489c-bf85-295f8f61fc20 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: d25ad8a1-1f52-4e6b-93a5-629420a06e66] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1845.579786] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Took 0.12 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1845.592108] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1845.592311] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1845.592480] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1845.621033] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1845.621252] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1845.621420] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1845.621773] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1845.621960] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1845.622137] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1845.629619] nova-conductor[53810]: DEBUG nova.quota [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Getting quotas for project e839303682f748a4b5a42c8a9273e388. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1845.631931] nova-conductor[53810]: DEBUG nova.quota [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Getting quotas for user cf4545757716483485ca9b60bd689a1c and project e839303682f748a4b5a42c8a9273e388. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1845.637231] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 4cbfa0ff-1a92-4d02-9929-b05931142d19] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1845.637615] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1845.637809] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1845.637975] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1845.640704] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 4cbfa0ff-1a92-4d02-9929-b05931142d19] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1845.641329] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1845.641525] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1845.641693] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1845.654906] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1845.655144] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1845.655322] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1855.626151] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell0:SELECT=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1855.644308] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=19 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1855.651198] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=12,nova_api:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1855.654049] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=12,nova_api:DELETE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1858.872406] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=46,nova_cell1:UPDATE=13,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1859.151335] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=46,nova_cell1:UPDATE=16,nova_cell1:INSERT=4,nova_cell1:SAVEPOINT=3,nova_cell1:RELEASE=3 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1875.517396] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=6,nova_cell1:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1879.210021] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=7 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1899.568717] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 4cbfa0ff-1a92-4d02-9929-b05931142d19] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 4cbfa0ff-1a92-4d02-9929-b05931142d19 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1899.569302] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1899.569543] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4cbfa0ff-1a92-4d02-9929-b05931142d19.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4cbfa0ff-1a92-4d02-9929-b05931142d19. [ 1899.569882] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 4cbfa0ff-1a92-4d02-9929-b05931142d19] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4cbfa0ff-1a92-4d02-9929-b05931142d19. [ 1899.594338] nova-conductor[53809]: DEBUG nova.network.neutron [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 4cbfa0ff-1a92-4d02-9929-b05931142d19] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1899.613665] nova-conductor[53809]: DEBUG nova.network.neutron [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 4cbfa0ff-1a92-4d02-9929-b05931142d19] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1899.616822] nova-conductor[53809]: DEBUG nova.network.neutron [None req-d4896d1a-4e63-4ea3-a319-ae9ed9e0a795 tempest-ServersTestJSON-818634346 tempest-ServersTestJSON-818634346-project-member] [instance: 4cbfa0ff-1a92-4d02-9929-b05931142d19] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1905.842617] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Took 0.13 seconds to select destinations for 1 instance(s). {{(pid=53809) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1905.855335] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1905.855513] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1905.855667] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1905.896794] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1905.897042] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1905.897225] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1905.897593] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1905.897781] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1905.897947] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1905.908094] nova-conductor[53809]: DEBUG nova.quota [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Getting quotas for project 92ed23cb43494196a6a832149a981bb0. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1905.912083] nova-conductor[53809]: DEBUG nova.quota [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Getting quotas for user 07c678a871574174b2bfefac2f989bb9 and project 92ed23cb43494196a6a832149a981bb0. Resources: {'ram', 'cores', 'instances'} {{(pid=53809) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1905.922159] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] [instance: 8eeb384b-b799-4b21-894f-c265e7e1b25e] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53809) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1905.922591] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1905.924264] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1905.924264] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1905.927276] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] [instance: 8eeb384b-b799-4b21-894f-c265e7e1b25e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53809) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1905.928175] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1905.928175] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1905.928344] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1905.945234] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1905.945453] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1905.945625] nova-conductor[53809]: DEBUG oslo_concurrency.lockutils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53809) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1915.902276] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell0:SELECT=2 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1915.930518] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=9,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=20,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1915.939457] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1915.943020] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_api:SELECT=11,nova_api:DELETE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1919.475903] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=46,nova_cell1:UPDATE=16,nova_cell1:INSERT=4,nova_cell1:SAVEPOINT=3,nova_cell1:RELEASE=3 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1935.454372] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=6,nova_cell1:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1941.209410] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=55,nova_cell1:UPDATE=14,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1946.204468] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1959.310533] nova-conductor[53810]: ERROR nova.scheduler.utils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] [instance: 8eeb384b-b799-4b21-894f-c265e7e1b25e] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 8eeb384b-b799-4b21-894f-c265e7e1b25e was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 1959.311255] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Rescheduling: True {{(pid=53810) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1959.311525] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8eeb384b-b799-4b21-894f-c265e7e1b25e.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8eeb384b-b799-4b21-894f-c265e7e1b25e. [ 1959.311900] nova-conductor[53810]: WARNING nova.scheduler.utils [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] [instance: 8eeb384b-b799-4b21-894f-c265e7e1b25e] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8eeb384b-b799-4b21-894f-c265e7e1b25e. [ 1959.333567] nova-conductor[53810]: DEBUG nova.network.neutron [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] [instance: 8eeb384b-b799-4b21-894f-c265e7e1b25e] deallocate_for_instance() {{(pid=53810) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1959.382605] nova-conductor[53810]: DEBUG nova.network.neutron [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] [instance: 8eeb384b-b799-4b21-894f-c265e7e1b25e] Instance cache missing network info. {{(pid=53810) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1959.386157] nova-conductor[53810]: DEBUG nova.network.neutron [None req-c8789fcf-e984-4061-bffe-d4eac82fc2f4 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] [instance: 8eeb384b-b799-4b21-894f-c265e7e1b25e] Updating instance_info_cache with network_info: [] {{(pid=53810) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1961.854047] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Took 0.13 seconds to select destinations for 1 instance(s). {{(pid=53810) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1961.867257] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1961.867477] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1961.867650] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1961.901333] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1961.901578] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1961.901719] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1961.902100] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1961.902291] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1961.902457] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1961.910117] nova-conductor[53810]: DEBUG nova.quota [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Getting quotas for project 92ed23cb43494196a6a832149a981bb0. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1961.912578] nova-conductor[53810]: DEBUG nova.quota [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Getting quotas for user 07c678a871574174b2bfefac2f989bb9 and project 92ed23cb43494196a6a832149a981bb0. Resources: {'ram', 'cores', 'instances'} {{(pid=53810) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1961.917731] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] [instance: 9cc4b1c2-2582-4881-9572-0f261464eac3] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=53810) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1961.918128] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1961.918328] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1961.918498] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1961.922442] nova-conductor[53810]: DEBUG nova.conductor.manager [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] [instance: 9cc4b1c2-2582-4881-9572-0f261464eac3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a8b5e8e9-a03f-44eb-a8f4-56b1df9fce22',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=53810) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1961.923060] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1961.923259] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1961.923428] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1961.936083] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Acquiring lock "3368e979-e585-49c6-913f-77bff95fda6f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}} [ 1961.936309] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}} [ 1961.936504] nova-conductor[53810]: DEBUG oslo_concurrency.lockutils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Lock "3368e979-e585-49c6-913f-77bff95fda6f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=53810) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}} [ 1971.906738] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell0:SELECT=2 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1971.925649] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=19 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1971.934504] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=15 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1971.935164] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1975.714612] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=48,nova_cell1:UPDATE=14,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1975.999212] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=48,nova_cell1:UPDATE=16,nova_cell1:INSERT=4,nova_cell1:SAVEPOINT=3,nova_cell1:RELEASE=3 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1996.419122] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=6,nova_cell1:UPDATE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 2002.208757] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=7 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 2014.911141] nova-conductor[53809]: ERROR nova.scheduler.utils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] [instance: 9cc4b1c2-2582-4881-9572-0f261464eac3] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2430, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2722, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 9cc4b1c2-2582-4881-9572-0f261464eac3 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 2014.911702] nova-conductor[53809]: DEBUG nova.conductor.manager [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Rescheduling: True {{(pid=53809) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 2014.911954] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9cc4b1c2-2582-4881-9572-0f261464eac3.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9cc4b1c2-2582-4881-9572-0f261464eac3. [ 2014.912950] nova-conductor[53809]: WARNING nova.scheduler.utils [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] [instance: 9cc4b1c2-2582-4881-9572-0f261464eac3] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9cc4b1c2-2582-4881-9572-0f261464eac3. [ 2014.949361] nova-conductor[53809]: DEBUG nova.network.neutron [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] [instance: 9cc4b1c2-2582-4881-9572-0f261464eac3] deallocate_for_instance() {{(pid=53809) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 2014.967184] nova-conductor[53809]: DEBUG nova.network.neutron [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] [instance: 9cc4b1c2-2582-4881-9572-0f261464eac3] Instance cache missing network info. {{(pid=53809) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 2014.970144] nova-conductor[53809]: DEBUG nova.network.neutron [None req-7776e6cb-ec13-4a4a-8313-5c860fc2b822 tempest-AttachVolumeTestJSON-1078261084 tempest-AttachVolumeTestJSON-1078261084-project-member] [instance: 9cc4b1c2-2582-4881-9572-0f261464eac3] Updating instance_info_cache with network_info: [] {{(pid=53809) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 2024.921193] nova-conductor[53810]: DEBUG dbcounter [-] [53810] Writing DB stats nova_cell1:SELECT=19,nova_cell1:UPDATE=8,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53810) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 2024.979989] nova-conductor[53809]: DEBUG dbcounter [-] [53809] Writing DB stats nova_cell1:SELECT=18,nova_cell1:UPDATE=7,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=53809) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}}