[ 406.738732] nova-conductor[53653]: INFO dbcounter [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Registered counter for database nova_api [ 406.748035] nova-conductor[53653]: DEBUG oslo_db.sqlalchemy.engines [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=53653) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 406.776729] nova-conductor[53653]: DEBUG dbcounter [-] [53653] Writer thread running {{(pid=53653) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 406.779038] nova-conductor[53653]: DEBUG nova.context [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),89980bcd-ea65-4338-957c-d9d250569837(cell1) {{(pid=53653) load_cells /opt/stack/nova/nova/context.py:464}} [ 406.780858] nova-conductor[53653]: DEBUG oslo_concurrency.lockutils [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53653) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 406.781204] nova-conductor[53653]: DEBUG oslo_concurrency.lockutils [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53653) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 406.781759] nova-conductor[53653]: DEBUG oslo_concurrency.lockutils [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53653) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 406.782297] nova-conductor[53653]: DEBUG oslo_concurrency.lockutils [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=53653) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 406.782610] nova-conductor[53653]: DEBUG oslo_concurrency.lockutils [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=53653) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 406.783654] nova-conductor[53653]: DEBUG oslo_concurrency.lockutils [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=53653) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 406.785131] nova-conductor[53653]: INFO dbcounter [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Registered counter for database nova_cell0 [ 406.787525] nova-conductor[53653]: INFO dbcounter [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Registered counter for database nova_cell1 [ 406.790231] nova-conductor[53653]: DEBUG oslo_db.sqlalchemy.engines [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=53653) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 406.790811] nova-conductor[53653]: DEBUG oslo_db.sqlalchemy.engines [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=53653) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 406.866987] nova-conductor[53653]: DEBUG dbcounter [-] [53653] Writer thread running {{(pid=53653) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 406.870554] nova-conductor[53653]: DEBUG dbcounter [-] [53653] Writer thread running {{(pid=53653) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 406.872342] nova-conductor[53653]: DEBUG oslo_concurrency.lockutils [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Acquiring lock "singleton_lock" {{(pid=53653) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 406.872629] nova-conductor[53653]: DEBUG oslo_concurrency.lockutils [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Acquired lock "singleton_lock" {{(pid=53653) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 406.872961] nova-conductor[53653]: DEBUG oslo_concurrency.lockutils [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Releasing lock "singleton_lock" {{(pid=53653) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 406.873448] nova-conductor[53653]: INFO oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Starting 2 workers [ 406.877593] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Started child 54074 {{(pid=53653) _start_child /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/service.py:579}} [ 406.881231] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Started child 54075 {{(pid=53653) _start_child /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/service.py:579}} [ 406.881739] nova-conductor[54074]: INFO nova.service [-] Starting conductor node (version 0.0.1) [ 406.882278] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Full set of CONF: {{(pid=53653) wait /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/service.py:653}} [ 406.882579] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ******************************************************************************** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2600}} [ 406.882829] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] Configuration options gathered from: {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2601}} [ 406.883101] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] command line args: ['--config-file', '/etc/nova/nova.conf'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}} [ 406.883531] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] config files: ['/etc/nova/nova.conf'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2603}} [ 406.883771] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ================================================================================ {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2605}} [ 406.884406] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] allow_resize_to_same_host = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.884744] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] arq_binding_timeout = 300 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.885092] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] block_device_allocate_retries = 60 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.886124] nova-conductor[54075]: INFO nova.service [-] Starting conductor node (version 0.0.1) [ 406.886528] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] block_device_allocate_retries_interval = 3 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.886851] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cert = self.pem {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.887148] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] compute_driver = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.887482] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] compute_monitors = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.887993] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] config_dir = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.888416] nova-conductor[54074]: INFO dbcounter [None req-c9ee924f-f398-4dbb-b0e4-2adcff211c7d None None] Registered counter for database nova_cell1 [ 406.888633] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] config_drive_format = iso9660 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.888888] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] config_file = ['/etc/nova/nova.conf'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.889205] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] config_source = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.889499] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] console_host = devstack {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.889811] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] control_exchange = nova {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.890109] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cpu_allocation_ratio = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.890398] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] daemon = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.890687] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] debug = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.890972] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] default_access_ip_network_name = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.891273] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] default_availability_zone = nova {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.891549] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] default_ephemeral_format = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.891827] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] default_green_pool_size = 1000 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.892248] nova-conductor[54075]: INFO dbcounter [None req-f33ebc5a-43b2-43e9-bc29-da53ceacc286 None None] Registered counter for database nova_cell1 [ 406.892484] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.892835] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] default_schedule_zone = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.893131] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] disk_allocation_ratio = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.893424] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] enable_new_services = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.893752] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] enabled_apis = ['osapi_compute'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.894056] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] enabled_ssl_apis = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.894354] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] flat_injected = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.894633] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] force_config_drive = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.895120] nova-conductor[54074]: DEBUG oslo_db.sqlalchemy.engines [None req-c9ee924f-f398-4dbb-b0e4-2adcff211c7d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=54074) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 406.895351] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] force_raw_images = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.895639] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] graceful_shutdown_timeout = 5 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.895930] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] heal_instance_info_cache_interval = 60 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.896527] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] host = devstack {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.896853] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] initial_cpu_allocation_ratio = 4.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.897148] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] initial_disk_allocation_ratio = 1.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.897450] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] initial_ram_allocation_ratio = 1.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.897909] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.898242] nova-conductor[54075]: DEBUG oslo_db.sqlalchemy.engines [None req-f33ebc5a-43b2-43e9-bc29-da53ceacc286 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=54075) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 406.898452] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] instance_build_timeout = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.898729] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] instance_delete_interval = 300 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.899033] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] instance_format = [instance: %(uuid)s] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.899313] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] instance_name_template = instance-%08x {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.899606] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] instance_usage_audit = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.899925] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] instance_usage_audit_period = month {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.900244] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] instance_uuid_format = [instance: %(uuid)s] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.900536] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] instances_path = /opt/stack/data/nova/instances {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.900817] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] internal_service_availability_zone = internal {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.901084] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] key = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.901385] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] live_migration_retry_count = 30 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.901680] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] log_color = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.901972] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] log_config_append = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.902261] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.902528] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] log_dir = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.902808] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] log_file = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.903053] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] log_options = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.903319] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] log_rotate_interval = 1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.903621] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] log_rotate_interval_type = days {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.903911] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] log_rotation_type = none {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.904162] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.904399] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.904680] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.904977] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.905210] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.905503] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] long_rpc_timeout = 1800 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.905857] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] max_concurrent_builds = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.906172] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writer thread running {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 406.906382] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] max_concurrent_live_migrations = 1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.906659] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] max_concurrent_snapshots = 5 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.906931] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] max_local_block_devices = 3 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.907199] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] max_logfile_count = 30 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.907940] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] max_logfile_size_mb = 200 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.908362] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] maximum_instance_delete_attempts = 5 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.908565] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] metadata_listen = 0.0.0.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.908801] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] metadata_listen_port = 8775 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.908996] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] metadata_workers = 2 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.909195] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writer thread running {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 406.909340] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] migrate_max_retries = -1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.909532] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] mkisofs_cmd = genisoimage {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.909753] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] my_block_storage_ip = 10.180.1.21 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.909918] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] my_ip = 10.180.1.21 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.910103] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] network_allocate_retries = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.910286] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.910472] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] osapi_compute_listen = 0.0.0.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.910639] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] osapi_compute_listen_port = 8774 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.910809] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] osapi_compute_unique_server_name_scope = {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.910978] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] osapi_compute_workers = 2 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.911140] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] password_length = 12 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.911298] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] periodic_enable = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.911465] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] periodic_fuzzy_delay = 60 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.911629] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] pointer_model = usbtablet {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.911825] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] preallocate_images = none {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.911988] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] publish_errors = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.912132] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] pybasedir = /opt/stack/nova {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.912289] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ram_allocation_ratio = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.912470] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] rate_limit_burst = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.912640] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] rate_limit_except_level = CRITICAL {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.912797] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] rate_limit_interval = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.912952] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] reboot_timeout = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.913108] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] reclaim_instance_interval = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.913280] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] record = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.913437] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] reimage_timeout_per_gb = 20 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.913603] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] report_interval = 120 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.913871] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] rescue_timeout = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.914186] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] reserved_host_cpus = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.914437] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] reserved_host_disk_mb = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.914652] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] reserved_host_memory_mb = 512 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.914916] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] reserved_huge_pages = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.915077] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] resize_confirm_window = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.915306] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] resize_fs_using_block_device = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.915521] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] resume_guests_state_on_host_boot = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.915933] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.916254] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] rpc_response_timeout = 60 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.916509] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] run_external_periodic_tasks = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.916725] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] running_deleted_instance_action = reap {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.916908] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] running_deleted_instance_poll_interval = 1800 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.917129] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] running_deleted_instance_timeout = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.917302] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] scheduler_instance_sync_interval = 120 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.917481] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] service_down_time = 720 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.917683] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] servicegroup_driver = db {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.917999] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] shelved_offload_time = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.918211] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] shelved_poll_interval = 3600 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.918416] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] shutdown_timeout = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.918620] nova-conductor[54074]: DEBUG nova.service [None req-c9ee924f-f398-4dbb-b0e4-2adcff211c7d None None] Creating RPC server for service conductor {{(pid=54074) start /opt/stack/nova/nova/service.py:186}} [ 406.918838] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] source_is_ipv6 = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.919049] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ssl_only = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.919247] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] state_path = /opt/stack/data/nova {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.919418] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] sync_power_state_interval = 600 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.919585] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] sync_power_state_pool_size = 1000 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.919759] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] syslog_log_facility = LOG_USER {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.919917] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] tempdir = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.920204] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] timeout_nbd = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.920385] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] transport_url = **** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.920547] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] update_resources_interval = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.920710] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] use_cow_images = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.920872] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] use_eventlog = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.921027] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] use_journal = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.921181] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] use_json = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.921341] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] use_rootwrap_daemon = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.921494] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] use_stderr = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.921644] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] use_syslog = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.921905] nova-conductor[54075]: DEBUG nova.service [None req-f33ebc5a-43b2-43e9-bc29-da53ceacc286 None None] Creating RPC server for service conductor {{(pid=54075) start /opt/stack/nova/nova/service.py:186}} [ 406.922055] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vcpu_pin_set = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.922243] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vif_plugging_is_fatal = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.922413] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vif_plugging_timeout = 300 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.922607] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] virt_mkfs = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.922774] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] volume_usage_poll_interval = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.922968] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] watch_log_file = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.923160] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] web = /usr/share/spice-html5 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}} [ 406.923427] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_concurrency.disable_process_locking = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.923632] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.923825] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.924015] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.924202] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_metrics.metrics_process_name = {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.924369] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.924532] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.924755] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.auth_strategy = keystone {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.924941] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.compute_link_prefix = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.925135] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.925312] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.dhcp_domain = novalocal {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.925483] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.enable_instance_password = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.925657] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.glance_link_prefix = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.925862] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.instance_list_cells_batch_fixed_size = 100 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.926056] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.instance_list_cells_batch_strategy = distributed {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.926222] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.instance_list_per_project_cells = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.926381] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.list_records_by_skipping_down_cells = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.926542] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.local_metadata_per_cell = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.926711] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.max_limit = 1000 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.926874] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.metadata_cache_expiration = 15 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.927051] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.neutron_default_tenant_id = default {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.927216] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.response_validation = warn {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.927376] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.use_neutron_default_nets = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.927562] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.vendordata_dynamic_connect_timeout = 5 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.927723] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.vendordata_dynamic_failure_fatal = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.927956] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.vendordata_dynamic_read_timeout = 5 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.928152] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.vendordata_dynamic_ssl_certfile = {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.928325] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.vendordata_dynamic_targets = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.928498] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.vendordata_jsonfile_path = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.928682] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api.vendordata_providers = ['StaticJSON'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.928984] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.backend = dogpile.cache.memcached {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.929179] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.backend_argument = **** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.929367] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.config_prefix = cache.oslo {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.929561] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.dead_timeout = 60.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.929741] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.debug_cache_backend = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.929978] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.enable_retry_client = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.930105] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.enable_socket_keepalive = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.930264] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.enabled = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.930423] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.enforce_fips_mode = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.930620] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.expiration_time = 600 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.930780] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.hashclient_retry_attempts = 2 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.930962] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.hashclient_retry_delay = 1.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.931140] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.memcache_dead_retry = 300 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.931301] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.memcache_password = **** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.931462] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.931620] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.931802] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.memcache_pool_maxsize = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.931984] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.932166] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.memcache_sasl_enabled = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.932341] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.memcache_servers = ['localhost:11211'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.932522] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.memcache_socket_timeout = 1.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.932692] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.memcache_username = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.932864] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.proxies = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.933047] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.redis_db = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.933205] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.redis_password = **** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.933389] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.redis_sentinel_service_name = mymaster {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.933565] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.redis_sentinels = ['localhost:26379'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.933730] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.redis_server = localhost:6379 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.933890] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.redis_socket_timeout = 1.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.934049] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.redis_username = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.934207] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.retry_attempts = 2 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.934372] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.retry_delay = 0.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.934533] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.socket_keepalive_count = 1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.934693] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.socket_keepalive_idle = 1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.934903] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.socket_keepalive_interval = 1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.935072] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.tls_allowed_ciphers = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.935228] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.tls_cafile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.935382] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.tls_certfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.935539] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.tls_enabled = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.935694] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cache.tls_keyfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.935912] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cinder.auth_section = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.936165] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cinder.auth_type = password {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.936370] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cinder.cafile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.936502] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cinder.catalog_info = volumev3::publicURL {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.936663] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cinder.certfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.936826] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cinder.collect_timing = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.936986] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cinder.cross_az_attach = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.937149] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cinder.debug = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.937326] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cinder.endpoint_template = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.937485] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cinder.http_retries = 3 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.937648] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cinder.insecure = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.937856] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cinder.keyfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.938052] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cinder.os_region_name = RegionOne {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.938226] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cinder.split_loggers = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.938386] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cinder.timeout = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.938579] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] compute.consecutive_build_service_disable_threshold = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.938748] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] compute.cpu_dedicated_set = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.938933] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] compute.cpu_shared_set = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.939129] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] compute.image_type_exclude_list = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.939302] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] compute.live_migration_wait_for_vif_plug = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.939465] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] compute.max_concurrent_disk_ops = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.939622] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] compute.max_disk_devices_to_attach = -1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.939785] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.939959] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.940146] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] compute.resource_provider_association_refresh = 300 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.940557] nova-conductor[54074]: INFO oslo.messaging._drivers.impl_rabbit [None req-c9ee924f-f398-4dbb-b0e4-2adcff211c7d None None] Creating fanout queue: conductor_fanout_b300316c4876472899258975de410073 [ 406.940702] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] compute.sharing_providers_max_uuids_per_request = 200 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.940886] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] compute.shutdown_retry_interval = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.941076] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] compute.vmdk_allowed_types = ['streamOptimized', 'monolithicSparse'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.941272] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] conductor.workers = 2 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.941456] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] console.allowed_origins = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.941616] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] console.ssl_ciphers = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.941789] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] console.ssl_minimum_version = default {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.941960] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] consoleauth.enforce_session_timeout = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.942134] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] consoleauth.token_ttl = 600 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.942329] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.cafile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.942485] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.certfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.942648] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.collect_timing = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.942815] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.connect_retries = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.942999] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.connect_retry_delay = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.943159] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.endpoint_override = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.943325] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.insecure = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.943484] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.keyfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.943643] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.max_version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.943805] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.min_version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.943964] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.region_name = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.944138] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.retriable_status_codes = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.944298] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.service_name = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.944486] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.service_type = accelerator {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.944653] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.split_loggers = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.944814] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.status_code_retries = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.944976] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.status_code_retry_delay = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.945136] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.timeout = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.945556] nova-conductor[54075]: INFO oslo.messaging._drivers.impl_rabbit [None req-f33ebc5a-43b2-43e9-bc29-da53ceacc286 None None] Creating fanout queue: conductor_fanout_fd246c26b21a43b8b4c2de038649b3a7 [ 406.945720] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.945910] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] cyborg.version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.947445] nova-conductor[54074]: DEBUG nova.service [None req-c9ee924f-f398-4dbb-b0e4-2adcff211c7d None None] Join ServiceGroup membership for this service conductor {{(pid=54074) start /opt/stack/nova/nova/service.py:203}} [ 406.947616] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.backend = sqlalchemy {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.947842] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.connection = **** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.948083] nova-conductor[54074]: DEBUG nova.servicegroup.drivers.db [None req-c9ee924f-f398-4dbb-b0e4-2adcff211c7d None None] DB_Driver: join new ServiceGroup member devstack to the conductor group, service = {{(pid=54074) join /opt/stack/nova/nova/servicegroup/drivers/db.py:44}} [ 406.948285] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.connection_debug = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.948486] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.connection_parameters = {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.948658] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.connection_recycle_time = 3600 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.948824] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.connection_trace = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.949062] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.db_inc_retry_interval = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.949251] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.db_max_retries = 20 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.949509] nova-conductor[54075]: DEBUG nova.service [None req-f33ebc5a-43b2-43e9-bc29-da53ceacc286 None None] Join ServiceGroup membership for this service conductor {{(pid=54075) start /opt/stack/nova/nova/service.py:203}} [ 406.949650] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.db_max_retry_interval = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.949798] nova-conductor[54075]: DEBUG nova.servicegroup.drivers.db [None req-f33ebc5a-43b2-43e9-bc29-da53ceacc286 None None] DB_Driver: join new ServiceGroup member devstack to the conductor group, service = {{(pid=54075) join /opt/stack/nova/nova/servicegroup/drivers/db.py:44}} [ 406.949933] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.db_retry_interval = 1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.950117] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.max_overflow = 50 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.950285] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.max_pool_size = 5 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.950449] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.max_retries = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.950618] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.mysql_sql_mode = TRADITIONAL {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.950777] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.mysql_wsrep_sync_wait = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.950936] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.pool_timeout = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.951098] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.retry_interval = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.951254] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.slave_connection = **** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.951412] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.sqlite_synchronous = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.951577] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] database.use_db_reconnect = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.951774] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.backend = sqlalchemy {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.951949] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.connection = **** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.952179] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.connection_debug = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.952362] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.connection_parameters = {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.952525] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.connection_recycle_time = 3600 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.952687] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.connection_trace = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.952847] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.db_inc_retry_interval = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.953009] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.db_max_retries = 20 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.953166] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.db_max_retry_interval = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.953323] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.db_retry_interval = 1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.953481] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.max_overflow = 50 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.953639] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.max_pool_size = 5 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.953797] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.max_retries = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.953964] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.mysql_sql_mode = TRADITIONAL {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.954120] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.mysql_wsrep_sync_wait = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.954275] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.pool_timeout = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.954430] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.retry_interval = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.954583] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.slave_connection = **** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.954738] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] api_database.sqlite_synchronous = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.954937] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] devices.enabled_mdev_types = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.955214] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.955439] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ephemeral_storage_encryption.default_format = luks {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.955610] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ephemeral_storage_encryption.enabled = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.955793] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ephemeral_storage_encryption.key_size = 512 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.955995] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.api_servers = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.956187] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.cafile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.956365] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.certfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.956528] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.collect_timing = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.956702] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.connect_retries = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.956859] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.connect_retry_delay = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.957040] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.debug = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.957204] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.default_trusted_certificate_ids = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.957364] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.enable_certificate_validation = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.957700] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.enable_rbd_download = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.957700] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.endpoint_override = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.957874] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.insecure = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.958046] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.keyfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.958203] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.max_version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.958357] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.min_version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.958519] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.num_retries = 3 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.958690] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.rbd_ceph_conf = {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.958877] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.rbd_connect_timeout = 5 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.959059] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.rbd_pool = {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.959226] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.rbd_user = {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.959385] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.region_name = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.959543] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.retriable_status_codes = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.959698] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.service_name = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.959870] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.service_type = image {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.960133] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.split_loggers = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.960221] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.status_code_retries = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.960363] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.status_code_retry_delay = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.960516] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.timeout = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.960713] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.valid_interfaces = ['internal', 'public'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.960881] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.verify_glance_signatures = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.961108] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] glance.version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.961310] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] guestfs.debug = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.961508] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] mks.enabled = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.962090] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.962306] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] image_cache.manager_interval = 2400 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.962496] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] image_cache.precache_concurrency = 1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.962672] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] image_cache.remove_unused_base_images = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.962839] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.963010] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.963192] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] image_cache.subdirectory_name = _base {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.963380] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.api_max_retries = 60 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.963545] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.api_retry_interval = 2 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.963725] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.auth_section = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.963888] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.auth_type = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.964059] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.cafile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.964223] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.certfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.964393] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.collect_timing = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.964574] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.conductor_group = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.964737] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.connect_retries = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.964895] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.connect_retry_delay = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.965057] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.endpoint_override = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.965217] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.insecure = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.965371] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.keyfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.965524] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.max_version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.965690] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.min_version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.965872] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.peer_list = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.966030] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.region_name = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.966185] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.retriable_status_codes = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.966345] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.serial_console_state_timeout = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.966500] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.service_name = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.966687] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.service_type = baremetal {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.966847] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.shard = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.967011] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.split_loggers = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.967170] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.status_code_retries = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.967332] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.status_code_retry_delay = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.967490] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.timeout = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.967668] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.valid_interfaces = ['internal', 'public'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.967853] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ironic.version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.968058] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.968258] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] key_manager.fixed_key = **** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.968468] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.968702] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican.barbican_api_version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.968941] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican.barbican_endpoint = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.969207] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican.barbican_endpoint_type = public {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.969467] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican.barbican_region_name = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.969705] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican.cafile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.969930] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican.certfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.970185] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican.collect_timing = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.970410] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican.insecure = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.970638] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican.keyfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.970879] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican.number_of_retries = 60 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.971113] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican.retry_delay = 1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.971354] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican.send_service_user_token = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.971591] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican.split_loggers = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.971820] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican.timeout = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.972072] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican.verify_ssl = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.972308] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican.verify_ssl_path = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.972572] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican_service_user.auth_section = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.972808] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican_service_user.auth_type = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.973073] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican_service_user.cafile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.973275] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican_service_user.certfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.973506] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican_service_user.collect_timing = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.973760] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican_service_user.insecure = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.973990] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican_service_user.keyfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.974223] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican_service_user.split_loggers = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.974456] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] barbican_service_user.timeout = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.974667] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vault.approle_role_id = **** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.974833] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vault.approle_secret_id = **** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.975012] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vault.kv_mountpoint = secret {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.975177] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vault.kv_path = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.975342] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vault.kv_version = 2 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.975503] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vault.namespace = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.975663] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vault.root_token_id = **** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.975819] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vault.ssl_ca_crt_file = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.975985] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vault.timeout = 60.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.976168] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vault.use_ssl = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.976359] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vault.vault_url = http://127.0.0.1:8200 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.976555] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.auth_section = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.976718] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.auth_type = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.976874] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.cafile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.977050] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.certfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.977218] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.collect_timing = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.977379] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.connect_retries = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.977552] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.connect_retry_delay = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.977723] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.endpoint_override = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.977908] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.insecure = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.978073] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.keyfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.978229] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.max_version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.978383] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.min_version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.978537] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.region_name = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.978693] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.retriable_status_codes = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.978848] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.service_name = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.979037] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.service_type = identity {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.979196] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.split_loggers = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.979351] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.status_code_retries = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.979506] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.status_code_retry_delay = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.979661] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.timeout = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.979836] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.valid_interfaces = ['internal', 'public'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.979995] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] keystone.version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.980243] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.connection_uri = {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.980425] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.cpu_mode = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.980637] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.cpu_model_extra_flags = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.980809] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.cpu_models = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.980981] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.cpu_power_governor_high = performance {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.981146] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.cpu_power_governor_low = powersave {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.981314] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.cpu_power_management = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.981527] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.cpu_power_management_strategy = cpu_state {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.981710] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.device_detach_attempts = 8 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.981888] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.device_detach_timeout = 20 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.982057] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.disk_cachemodes = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.982210] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.disk_prefix = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.982371] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.enabled_perf_events = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.982532] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.file_backed_memory = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.982690] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.gid_maps = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.982864] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.hw_disk_discard = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.983020] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.hw_machine_type = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.983183] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.images_rbd_ceph_conf = {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.983346] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.983525] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.983695] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.images_rbd_glance_store_name = {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.983860] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.images_rbd_pool = rbd {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.984063] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.images_type = default {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.984228] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.images_volume_group = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.984409] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.inject_key = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.984573] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.inject_partition = -2 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.984735] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.inject_password = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.984914] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.iscsi_iface = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.985078] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.iser_use_multipath = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.985240] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.live_migration_bandwidth = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.985399] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.live_migration_completion_timeout = 800 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.985557] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.live_migration_downtime = 500 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.985715] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.live_migration_downtime_delay = 75 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.985875] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.live_migration_downtime_steps = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.986030] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.live_migration_inbound_addr = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.986193] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.live_migration_permit_auto_converge = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.986351] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.live_migration_permit_post_copy = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.986505] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.live_migration_scheme = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.986671] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.live_migration_timeout_action = abort {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.986829] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.live_migration_tunnelled = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.986984] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.live_migration_uri = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.987142] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.live_migration_with_native_tls = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.987323] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.max_queues = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.987482] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.mem_stats_period_seconds = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.987729] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.migration_inbound_addr = 10.180.1.21 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.987901] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.nfs_mount_options = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.988148] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.988322] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.num_aoe_discover_tries = 3 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.988486] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.num_iser_scan_tries = 5 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.988645] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.num_memory_encrypted_guests = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.988807] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.num_nvme_discover_tries = 5 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.988967] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.num_pcie_ports = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.989128] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.num_volume_scan_tries = 5 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.989289] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.pmem_namespaces = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.989449] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.quobyte_client_cfg = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.989663] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.989827] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.rbd_connect_timeout = 5 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.989991] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.rbd_destroy_volume_retries = 12 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.990151] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.990307] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.rbd_secret_uuid = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.990461] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.rbd_user = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.990657] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.realtime_scheduler_priority = 1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.990850] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.remote_filesystem_transport = ssh {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.991013] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.rescue_image_id = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.991170] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.rescue_kernel_id = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.991321] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.rescue_ramdisk_id = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.991484] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.rng_dev_path = /dev/urandom {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.991640] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.rx_queue_size = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.991805] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.smbfs_mount_options = {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.992020] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.992188] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.snapshot_compression = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.992344] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.snapshot_image_format = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.992554] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.992716] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.sparse_logical_volumes = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.992878] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.swtpm_enabled = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.993066] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.swtpm_group = tss {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.993230] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.swtpm_user = tss {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.993395] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.sysinfo_serial = unique {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.993551] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.tb_cache_size = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.993703] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.tx_queue_size = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.993866] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.uid_maps = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.994027] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.use_virtio_for_bridges = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.994193] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.virt_type = kvm {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.994357] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.volume_clear = zero {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.994519] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.volume_clear_size = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.994679] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.volume_use_multipath = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.994851] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.vzstorage_cache_path = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.995020] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.995187] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.vzstorage_mount_group = qemu {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.995346] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.vzstorage_mount_opts = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.995507] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.vzstorage_mount_perms = 0770 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.995713] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.995878] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.vzstorage_mount_user = stack {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.996056] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] libvirt.wait_soft_reboot_seconds = 120 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.996232] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.auth_section = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.996420] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.auth_type = password {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.996577] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.cafile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.996736] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.certfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.996894] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.collect_timing = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.997072] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.connect_retries = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.997228] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.connect_retry_delay = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.997422] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.default_floating_pool = public {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.997581] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.endpoint_override = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.997741] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.extension_sync_interval = 600 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.997903] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.http_retries = 3 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.998060] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.insecure = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.998236] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.keyfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.998391] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.max_version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.998558] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.metadata_proxy_shared_secret = **** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.998713] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.min_version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.998880] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.ovs_bridge = br-int {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.999093] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.physnets = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.999263] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.region_name = RegionOne {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.999422] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.retriable_status_codes = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.999588] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.service_metadata_proxy = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.999744] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.service_name = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 406.999911] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.service_type = network {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.000085] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.split_loggers = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.000247] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.status_code_retries = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.000406] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.status_code_retry_delay = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.000562] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.timeout = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.000751] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.valid_interfaces = ['internal', 'public'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.000912] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] neutron.version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.001085] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] notifications.bdms_in_notifications = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.001264] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] notifications.default_level = INFO {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.001435] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] notifications.notification_format = unversioned {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.001593] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] notifications.notify_on_state_change = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.001763] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.001937] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] pci.alias = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.002101] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] pci.device_spec = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.002264] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] pci.report_in_placement = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.002468] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.auth_section = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.002642] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.auth_type = password {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.002828] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.auth_url = http://10.180.1.21/identity {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.002989] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.cafile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.003145] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.certfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.003307] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.collect_timing = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.003464] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.connect_retries = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.003621] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.connect_retry_delay = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.003783] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.default_domain_id = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.003941] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.default_domain_name = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.004109] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.domain_id = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.004267] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.domain_name = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.004423] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.endpoint_override = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.004583] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.insecure = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.004738] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.keyfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.004894] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.max_version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.005049] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.min_version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.005215] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.password = **** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.005396] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.project_domain_id = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.005562] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.project_domain_name = Default {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.005771] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.project_id = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.005951] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.project_name = service {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.006115] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.region_name = RegionOne {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.006274] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.retriable_status_codes = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.006433] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.service_name = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.006599] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.service_type = placement {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.006761] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.split_loggers = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.006925] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.status_code_retries = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.007082] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.status_code_retry_delay = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.007240] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.system_scope = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.007396] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.timeout = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.007551] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.trust_id = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.007709] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.user_domain_id = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.007892] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.user_domain_name = Default {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.008082] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.user_id = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.008260] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.username = nova {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.008441] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.valid_interfaces = ['internal', 'public'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.008602] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] placement.version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.008797] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] quota.cores = 20 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.008965] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] quota.count_usage_from_placement = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.009133] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] quota.driver = nova.quota.DbQuotaDriver {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.009320] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] quota.injected_file_content_bytes = 10240 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.009487] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] quota.injected_file_path_length = 255 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.009671] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] quota.injected_files = 5 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.009837] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] quota.instances = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.010002] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] quota.key_pairs = 100 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.010167] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] quota.metadata_items = 128 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.010329] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] quota.ram = 51200 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.010488] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] quota.recheck_quota = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.010650] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] quota.server_group_members = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.010814] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] quota.server_groups = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.011199] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.011199] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] scheduler.enable_isolated_aggregate_filtering = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.011373] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] scheduler.image_metadata_prefilter = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.011550] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.011717] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] scheduler.max_attempts = 3 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.011879] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] scheduler.max_placement_results = 1000 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.012077] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] scheduler.placement_aggregate_required_for_tenants = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.012244] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] scheduler.query_placement_for_image_type_support = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.012430] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.012604] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] scheduler.workers = 2 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.012805] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.012979] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.013174] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.013344] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.013507] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.013670] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.013831] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.014040] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.enabled_filters = ['ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.014215] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.host_subset_size = 1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.014383] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.hypervisor_version_weight_multiplier = 1.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.014558] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.image_properties_default_architecture = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.014725] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.014886] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.isolated_hosts = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.015049] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.isolated_images = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.015211] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.max_instances_per_host = 50 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.015369] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.max_io_ops_per_host = 8 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.015529] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.num_instances_weight_multiplier = 0.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.015694] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.pci_in_placement = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.015849] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.016016] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.016188] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.016347] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.016506] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.016664] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.016826] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.track_instance_changes = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.017000] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.017173] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] metrics.required = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.017334] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] metrics.weight_multiplier = 1.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.017492] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] metrics.weight_of_unavailable = -10000.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.017653] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] metrics.weight_setting = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.017986] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.018154] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] serial_console.enabled = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.018352] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] serial_console.port_range = 10000:20000 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.018522] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] serial_console.proxyclient_address = 127.0.0.1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.018688] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] serial_console.serialproxy_host = 0.0.0.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.018858] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] serial_console.serialproxy_port = 6083 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.019019] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] service_user.auth_section = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.019187] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] service_user.auth_type = password {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.019343] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] service_user.cafile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.019498] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] service_user.certfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.019657] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] service_user.collect_timing = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.019814] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] service_user.insecure = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.019973] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] service_user.keyfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.020157] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] service_user.send_service_user_token = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.020316] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] service_user.split_loggers = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.020471] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] service_user.timeout = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.020660] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] spice.agent_enabled = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.020820] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] spice.enabled = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.021163] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.021357] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] spice.html5proxy_host = 0.0.0.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.021525] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] spice.html5proxy_port = 6082 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.021710] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] spice.image_compression = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.021866] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] spice.jpeg_compression = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.022022] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] spice.playback_compression = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.022190] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] spice.server_listen = 127.0.0.1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.022354] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] spice.server_proxyclient_address = 127.0.0.1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.022510] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] spice.streaming_mode = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.022664] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] spice.zlib_compression = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.022847] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] upgrade_levels.baseapi = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.023020] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] upgrade_levels.compute = auto {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.023179] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] upgrade_levels.conductor = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.023354] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] upgrade_levels.scheduler = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.023520] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vendordata_dynamic_auth.auth_section = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.023679] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vendordata_dynamic_auth.auth_type = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.023834] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vendordata_dynamic_auth.cafile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.023989] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vendordata_dynamic_auth.certfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.024164] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vendordata_dynamic_auth.collect_timing = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.024325] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vendordata_dynamic_auth.insecure = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.024479] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vendordata_dynamic_auth.keyfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.024637] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vendordata_dynamic_auth.split_loggers = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.024793] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vendordata_dynamic_auth.timeout = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.025004] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.api_retry_count = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.025164] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.ca_file = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.025339] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.cache_prefix = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.025497] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.cluster_name = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.025671] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.connection_pool_size = 10 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.025816] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.console_delay_seconds = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.025975] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.datastore_regex = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.026134] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.host_ip = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.026292] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.host_password = **** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.026457] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.host_port = 443 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.026617] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.host_username = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.026776] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.insecure = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.026935] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.integration_bridge = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.027102] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.maximum_objects = 100 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.027260] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.pbm_default_policy = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.027420] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.pbm_enabled = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.027576] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.pbm_wsdl_location = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.027742] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.027913] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.serial_port_proxy_uri = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.028085] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.serial_port_service_uri = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.028254] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.task_poll_interval = 0.5 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.028417] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.use_linked_clone = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.028584] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.vnc_keymap = en-us {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.028749] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.vnc_port = 5900 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.028912] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vmware.vnc_port_total = 10000 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.029129] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vnc.auth_schemes = ['none'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.029300] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vnc.enabled = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.029631] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.029821] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vnc.novncproxy_host = 0.0.0.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.029994] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vnc.novncproxy_port = 6080 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.030176] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vnc.server_listen = 127.0.0.1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.030345] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.030502] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vnc.vencrypt_ca_certs = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.030657] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vnc.vencrypt_client_cert = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.030812] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] vnc.vencrypt_client_key = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.031024] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.disable_compute_service_check_for_ffu = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.031187] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.disable_deep_image_inspection = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.031352] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.disable_fallback_pcpu_query = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.031509] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.disable_group_policy_check_upcall = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.031665] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.disable_libvirt_livesnapshot = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.031821] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.disable_rootwrap = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.031979] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.enable_numa_live_migration = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.032163] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.enable_qemu_monitor_announce_self = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.032326] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.032485] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.handle_virt_lifecycle_events = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.032640] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.libvirt_disable_apic = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.032795] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.never_download_image_if_on_rbd = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.032955] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.qemu_monitor_announce_self_count = 3 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.033111] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.qemu_monitor_announce_self_interval = 1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.033271] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.033428] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.skip_cpu_compare_at_startup = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.033585] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.skip_cpu_compare_on_dest = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.033743] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.033900] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.skip_reserve_in_use_ironic_nodes = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.034058] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.034240] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.034438] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.034604] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] wsgi.client_socket_timeout = 900 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.034771] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] wsgi.default_pool_size = 1000 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.034936] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] wsgi.keep_alive = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.035101] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] wsgi.max_header_line = 16384 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.035260] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] wsgi.secure_proxy_ssl_header = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.035417] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] wsgi.ssl_ca_file = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.035571] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] wsgi.ssl_cert_file = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.035727] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] wsgi.ssl_key_file = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.035888] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] wsgi.tcp_keepidle = 600 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.036095] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.036270] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] zvm.ca_file = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.036430] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] zvm.cloud_connector_url = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.036653] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.036819] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] zvm.reachable_timeout = 300 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.037038] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_policy.enforce_new_defaults = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.037213] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_policy.enforce_scope = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.037403] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_policy.policy_default_rule = default {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.037578] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_policy.policy_dirs = ['policy.d'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.037750] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_policy.policy_file = policy.yaml {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.037953] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.038133] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.038291] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.038446] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.038607] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.038805] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.038989] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.039194] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] profiler.connection_string = messaging:// {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.039366] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] profiler.enabled = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.039551] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] profiler.es_doc_type = notification {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.039734] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] profiler.es_scroll_size = 10000 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.039902] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] profiler.es_scroll_time = 2m {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.040081] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] profiler.filter_error_trace = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.040250] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] profiler.hmac_keys = **** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.040416] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] profiler.sentinel_service_name = mymaster {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.040605] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] profiler.socket_timeout = 0.1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.040769] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] profiler.trace_requests = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.040932] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] profiler.trace_sqlalchemy = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.041141] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] profiler_jaeger.process_tags = {} {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.041329] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] profiler_jaeger.service_name_prefix = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.041498] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] profiler_otlp.service_name_prefix = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.041691] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] remote_debug.host = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.041896] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] remote_debug.port = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.042126] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.042299] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.042486] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.042648] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.042828] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.043005] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.043176] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.043351] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.043528] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.043698] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.hostname = devstack {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.043876] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.044075] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.044260] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.044433] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.044628] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.processname = nova-conductor {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.044817] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.044983] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.045151] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.045310] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.045467] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.045625] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.045790] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.045949] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.046112] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.046270] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.046427] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.046589] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.046745] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.046910] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.047080] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.ssl = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.047252] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.ssl_ca_file = {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.047416] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.ssl_cert_file = {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.047574] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.047742] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.ssl_key_file = {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.047909] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.ssl_version = {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.048093] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_rabbit.use_queue_manager = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.048324] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.048492] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_notifications.retry = -1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.048673] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.048848] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_messaging_notifications.transport_url = **** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.049057] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.auth_section = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.049222] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.auth_type = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.049378] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.cafile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.049532] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.certfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.049691] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.collect_timing = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.049846] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.connect_retries = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.050028] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.connect_retry_delay = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.050185] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.endpoint_id = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.050337] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.endpoint_override = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.050495] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.insecure = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.050649] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.keyfile = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.050801] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.max_version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.050955] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.min_version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.051106] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.region_name = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.051257] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.retriable_status_codes = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.051432] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.service_name = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.051585] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.service_type = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.051748] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.split_loggers = False {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.051905] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.status_code_retries = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.052082] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.status_code_retry_delay = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.052243] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.timeout = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.052422] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.valid_interfaces = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.052583] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_limit.version = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.052780] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_reports.file_event_handler = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.052972] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.053131] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] oslo_reports.log_dir = None {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}} [ 407.053258] nova-conductor[53653]: DEBUG oslo_service.service [None req-a19a7227-bfa1-4258-b176-ed24f3304ed9 None None] ******************************************************************************** {{(pid=53653) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2624}} [ 416.777618] nova-conductor[53653]: DEBUG dbcounter [-] [53653] Writing DB stats nova_api:SELECT=1 {{(pid=53653) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 416.868189] nova-conductor[53653]: DEBUG dbcounter [-] [53653] Writing DB stats nova_cell0:SELECT=1 {{(pid=53653) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 416.871344] nova-conductor[53653]: DEBUG dbcounter [-] [53653] Writing DB stats nova_cell1:SELECT=1 {{(pid=53653) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 421.958912] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=5,nova_cell1:INSERT=1,nova_cell1:UPDATE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 421.964771] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=4,nova_cell1:INSERT=1,nova_cell1:UPDATE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 494.177295] nova-conductor[54075]: DEBUG oslo_db.sqlalchemy.engines [None req-b1329809-3ee4-4bf6-b0f8-ca3ec5987b2e None None] Parent process 53653 forked (54075) with an open database connection, which is being discarded and recreated. {{(pid=54075) checkout /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:438}} [ 494.179575] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writer thread running {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 504.180253] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 512.921961] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=6,nova_cell1:INSERT=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 516.924980] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=1,nova_cell1:UPDATE=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 527.582722] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=2 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 527.589362] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Creating reply queue: reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 527.602206] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Expecting reply to msg 1921aa2a5f824219a7f949d7113b36ac in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 527.975123] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1921aa2a5f824219a7f949d7113b36ac [ 528.105285] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Took 0.52 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 528.134410] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 528.134696] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 528.136760] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 528.138760] nova-conductor[54075]: INFO dbcounter [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Registered counter for database nova_cell1 [ 528.141680] nova-conductor[54075]: DEBUG oslo_db.sqlalchemy.engines [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=54075) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 528.146874] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writer thread running {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 528.217053] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 528.217288] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 528.217746] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 528.218206] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 528.218388] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 528.218542] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 528.219933] nova-conductor[54075]: INFO dbcounter [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Registered counter for database nova_cell0 [ 528.229569] nova-conductor[54075]: DEBUG oslo_db.sqlalchemy.engines [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=54075) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 528.240493] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writer thread running {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 528.259558] nova-conductor[54075]: DEBUG nova.quota [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Getting quotas for project 23c75f6fd4fd411d86839e1ac7c8c961. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 528.265549] nova-conductor[54075]: DEBUG nova.quota [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Getting quotas for user c5b4439dac21460ea2f4479a3b0bc37c and project 23c75f6fd4fd411d86839e1ac7c8c961. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 528.270905] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] [instance: d27c729c-1991-41bc-b76f-9c519333330a] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 528.271477] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 528.271679] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 528.271842] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 528.277848] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] [instance: d27c729c-1991-41bc-b76f-9c519333330a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 528.278547] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 528.278753] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 528.278909] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 528.311368] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 528.311607] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 528.311777] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 528.312119] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Acquiring lock "compute-rpcapi-router" {{(pid=54075) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 528.312224] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Acquired lock "compute-rpcapi-router" {{(pid=54075) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 528.312761] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e610df73-64cb-49fb-9c30-3491f5572437 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 528.312931] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e610df73-64cb-49fb-9c30-3491f5572437 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 528.313084] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e610df73-64cb-49fb-9c30-3491f5572437 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 528.313421] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e610df73-64cb-49fb-9c30-3491f5572437 None None] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 528.313593] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e610df73-64cb-49fb-9c30-3491f5572437 None None] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 528.313742] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e610df73-64cb-49fb-9c30-3491f5572437 None None] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 528.320140] nova-conductor[54075]: INFO nova.compute.rpcapi [None req-e610df73-64cb-49fb-9c30-3491f5572437 None None] Automatically selected compute RPC version 6.3 from minimum service version 67 [ 528.320599] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e610df73-64cb-49fb-9c30-3491f5572437 None None] Releasing lock "compute-rpcapi-router" {{(pid=54075) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 538.304043] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=5,nova_api:UPDATE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 538.306864] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=6,nova_api:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 538.319015] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=2 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 538.319940] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=5,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=9,nova_cell1:RELEASE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 540.137689] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writer thread running {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 540.138467] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 540.144380] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Creating reply queue: reply_07d28983e5e94d2e89bfc63e924db1f9 [ 540.154290] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Expecting reply to msg 63e2e433dbdb495c872c0a0da683bd2b in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 540.398213] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 63e2e433dbdb495c872c0a0da683bd2b [ 540.660913] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Took 0.52 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 540.701740] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 540.701954] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 540.704610] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 540.707037] nova-conductor[54074]: INFO dbcounter [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Registered counter for database nova_cell1 [ 540.715917] nova-conductor[54074]: DEBUG oslo_db.sqlalchemy.engines [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=54074) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 540.722846] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writer thread running {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 540.813882] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 540.814136] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 540.814597] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 540.815062] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 540.815243] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 540.815397] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 540.816818] nova-conductor[54074]: INFO dbcounter [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Registered counter for database nova_cell0 [ 540.830677] nova-conductor[54074]: DEBUG oslo_db.sqlalchemy.engines [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=54074) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 540.841131] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writer thread running {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 540.854132] nova-conductor[54074]: DEBUG nova.quota [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Getting quotas for project 423f777bec3c474a91970fce3e308097. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 540.859710] nova-conductor[54074]: DEBUG nova.quota [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Getting quotas for user e4ea22bb21004f69a2b27d306493db45 and project 423f777bec3c474a91970fce3e308097. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 540.866040] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 67c74993-5779-48fb-9a8c-8904a857d22a] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 540.866641] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 540.866827] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 540.867087] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 540.879370] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 67c74993-5779-48fb-9a8c-8904a857d22a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 540.881034] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 540.881795] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 540.881795] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 540.934869] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 540.935220] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 540.935497] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 540.935883] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "compute-rpcapi-router" {{(pid=54074) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 540.936099] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquired lock "compute-rpcapi-router" {{(pid=54074) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 540.937061] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-38ac899e-ce8b-427f-b11d-7641112925dc None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 540.937061] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-38ac899e-ce8b-427f-b11d-7641112925dc None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 540.937231] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-38ac899e-ce8b-427f-b11d-7641112925dc None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 540.937875] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-38ac899e-ce8b-427f-b11d-7641112925dc None None] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 540.938324] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-38ac899e-ce8b-427f-b11d-7641112925dc None None] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 540.938855] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-38ac899e-ce8b-427f-b11d-7641112925dc None None] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 540.950547] nova-conductor[54074]: INFO nova.compute.rpcapi [None req-38ac899e-ce8b-427f-b11d-7641112925dc None None] Automatically selected compute RPC version 6.3 from minimum service version 67 [ 540.952105] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-38ac899e-ce8b-427f-b11d-7641112925dc None None] Releasing lock "compute-rpcapi-router" {{(pid=54074) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 541.762622] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Expecting reply to msg 1cb906acc7cb4d20b23924d8b4bc7c54 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 541.983169] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1cb906acc7cb4d20b23924d8b4bc7c54 [ 542.264743] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 542.277679] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 542.277928] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 542.278102] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 542.320524] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 542.320633] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 542.320765] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 542.321373] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 542.321373] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 542.321568] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 542.333669] nova-conductor[54074]: DEBUG nova.quota [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Getting quotas for project b0e5d54143934f498a4a7849ec99a645. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 542.337214] nova-conductor[54074]: DEBUG nova.quota [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Getting quotas for user f67ad126d9354c9e9cd8ac79b829fb4a and project b0e5d54143934f498a4a7849ec99a645. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 542.342210] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] [instance: 3888f958-a6ea-4159-b2be-1b2b5781f41f] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 542.342698] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 542.342907] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 542.343081] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 542.349411] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] [instance: 3888f958-a6ea-4159-b2be-1b2b5781f41f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 542.350205] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 542.350449] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 542.350650] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 542.371126] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 542.371649] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 542.371649] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 545.141776] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] [instance: d27c729c-1991-41bc-b76f-9c519333330a] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 89cd610f-a172-46da-825b-a2a0b6b1be90, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance d27c729c-1991-41bc-b76f-9c519333330a was re-scheduled: Binding failed for port 89cd610f-a172-46da-825b-a2a0b6b1be90, please check neutron logs for more information.\n'] [ 545.142540] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 545.142922] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d27c729c-1991-41bc-b76f-9c519333330a.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d27c729c-1991-41bc-b76f-9c519333330a. [ 545.143417] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] [instance: d27c729c-1991-41bc-b76f-9c519333330a] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d27c729c-1991-41bc-b76f-9c519333330a. [ 545.202209] nova-conductor[54075]: DEBUG nova.network.neutron [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] [instance: d27c729c-1991-41bc-b76f-9c519333330a] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 546.094992] nova-conductor[54075]: DEBUG nova.network.neutron [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] [instance: d27c729c-1991-41bc-b76f-9c519333330a] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 546.102411] nova-conductor[54075]: DEBUG nova.network.neutron [None req-58b2eba5-476b-4f74-88b6-9834b8ca2aab tempest-TenantUsagesTestJSON-1981865800 tempest-TenantUsagesTestJSON-1981865800-project-member] [instance: d27c729c-1991-41bc-b76f-9c519333330a] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 546.741584] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Expecting reply to msg f43a9cdd5cfc43d3891b7904505f2816 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 547.127898] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f43a9cdd5cfc43d3891b7904505f2816 [ 547.245648] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 547.267420] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 547.267661] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 547.267892] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 547.457870] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 547.458201] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 547.458280] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 547.458649] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 547.458857] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 547.459031] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 547.483248] nova-conductor[54074]: DEBUG nova.quota [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Getting quotas for project 4a15f084eff34d2b840889a457bdaeb3. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 547.493674] nova-conductor[54074]: DEBUG nova.quota [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Getting quotas for user 23ccfeaefb6445009a1e73e9e8c5d07b and project 4a15f084eff34d2b840889a457bdaeb3. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 547.500759] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] [instance: 834573d9-496f-4c80-b157-e1b12f799418] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 547.501171] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 547.501385] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 547.501553] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 547.640277] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] [instance: 834573d9-496f-4c80-b157-e1b12f799418] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 547.640691] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 547.640901] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 547.641067] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 547.787036] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 547.787293] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 547.788075] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 554.180478] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Expecting reply to msg c4b0b256c362477aa951a30c0e9988d4 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 554.636552] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c4b0b256c362477aa951a30c0e9988d4 [ 554.683909] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 554.720574] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 554.720848] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 554.721043] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 554.883889] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 554.884337] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 554.884611] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 554.885187] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 554.885470] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 554.885885] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 554.917299] nova-conductor[54074]: DEBUG nova.quota [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Getting quotas for project 2274ea1508fa4b67af0d4dd8497524b8. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 554.922587] nova-conductor[54074]: DEBUG nova.quota [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Getting quotas for user e43939b905d44b168a3f82f8158de1e2 and project 2274ea1508fa4b67af0d4dd8497524b8. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 554.929163] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] [instance: 9cbf81d9-92b5-4edf-8e7e-f4caa8f44f66] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 554.930003] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 554.930003] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 554.930175] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 554.953268] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] [instance: 9cbf81d9-92b5-4edf-8e7e-f4caa8f44f66] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 554.954076] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 554.954287] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 554.954452] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 555.079080] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 555.079352] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 555.079522] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 555.735037] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Expecting reply to msg f59c00c1039647809dd85bdcdafdb4b1 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 556.013565] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f59c00c1039647809dd85bdcdafdb4b1 [ 556.238265] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 556.272913] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 556.273239] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 556.273484] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 556.412756] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 556.413001] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 556.413181] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 556.413559] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 556.413745] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 556.413907] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 556.438993] nova-conductor[54075]: DEBUG nova.quota [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Getting quotas for project 19d0c5c1785f4cfda4bcfd4dd7e1a796. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 556.447956] nova-conductor[54075]: DEBUG nova.quota [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Getting quotas for user 123a7d0ce1ae4b5d9bb9563402f9f306 and project 19d0c5c1785f4cfda4bcfd4dd7e1a796. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 556.455496] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] [instance: 7bac3f71-a8a9-47fb-81c8-1b2b7fbdde8f] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 556.456142] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 556.456308] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 556.456488] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 556.470793] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] [instance: 7bac3f71-a8a9-47fb-81c8-1b2b7fbdde8f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 556.471535] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 556.471766] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 556.471975] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 556.582463] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 556.582809] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 556.582992] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-92b2ad58-7f69-4130-b419-4f54899a1ddc tempest-ServersAaction247Test-1038962410 tempest-ServersAaction247Test-1038962410-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 556.925080] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Expecting reply to msg 28c94883105549288cb49978ec6a88cd in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 556.968197] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Expecting reply to msg 1e1d79176f224e80a77ea70d22418985 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 557.274862] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 28c94883105549288cb49978ec6a88cd [ 557.340090] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1e1d79176f224e80a77ea70d22418985 [ 557.427399] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 557.461542] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 557.461790] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 557.461958] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 557.484282] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Took 0.52 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 557.497622] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 557.497919] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 557.498037] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 557.582281] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 557.582281] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 557.582281] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 557.582526] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 557.582705] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 557.582860] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 557.590435] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 557.590671] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 557.590904] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 557.591213] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 557.591403] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 557.591566] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 557.598874] nova-conductor[54075]: DEBUG nova.quota [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Getting quotas for project 61a866168186462d9d849072a1ff25f1. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 557.601880] nova-conductor[54075]: DEBUG nova.quota [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Getting quotas for user 6349e1aff7d945a6a471b1f4e826b23f and project 61a866168186462d9d849072a1ff25f1. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 557.607238] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 22179940-4e5b-4879-be19-a9addb0a628c] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 557.608163] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 557.608568] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 557.608997] nova-conductor[54074]: DEBUG nova.quota [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Getting quotas for project 2274ea1508fa4b67af0d4dd8497524b8. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 557.609247] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 557.612238] nova-conductor[54074]: DEBUG nova.quota [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Getting quotas for user e43939b905d44b168a3f82f8158de1e2 and project 2274ea1508fa4b67af0d4dd8497524b8. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 557.625388] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 22179940-4e5b-4879-be19-a9addb0a628c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 557.632954] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] [instance: 15dc4e5a-da5b-4657-8aec-f501d35d7a58] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 557.635356] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 557.635356] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 557.635356] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 557.636249] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 557.636700] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 557.637575] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 557.694749] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] [instance: 15dc4e5a-da5b-4657-8aec-f501d35d7a58] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 557.695551] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 557.695772] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 557.695974] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 557.749634] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 557.749968] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 557.750170] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 557.751546] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 557.751788] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 557.751954] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 558.652075] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Expecting reply to msg 419715d09d2f4896a15a9b9cf67b82cf in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 558.951859] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 67c74993-5779-48fb-9a8c-8904a857d22a] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 9411c776-3d75-4688-8670-d99f84c3a7e9, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 67c74993-5779-48fb-9a8c-8904a857d22a was re-scheduled: Binding failed for port 9411c776-3d75-4688-8670-d99f84c3a7e9, please check neutron logs for more information.\n'] [ 558.952488] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 558.952710] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 419715d09d2f4896a15a9b9cf67b82cf [ 558.953087] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 67c74993-5779-48fb-9a8c-8904a857d22a.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 67c74993-5779-48fb-9a8c-8904a857d22a. [ 558.953761] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 67c74993-5779-48fb-9a8c-8904a857d22a] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 67c74993-5779-48fb-9a8c-8904a857d22a. [ 559.015129] nova-conductor[54075]: DEBUG nova.network.neutron [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 67c74993-5779-48fb-9a8c-8904a857d22a] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 559.050800] nova-conductor[54075]: DEBUG nova.network.neutron [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 67c74993-5779-48fb-9a8c-8904a857d22a] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 559.056026] nova-conductor[54075]: DEBUG nova.network.neutron [None req-8daba062-bc3d-447a-8796-bdc3dbcb48bd tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 67c74993-5779-48fb-9a8c-8904a857d22a] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 559.155719] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 559.185450] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 559.185623] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 559.191792] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 559.400646] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 559.400646] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 559.400646] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 559.400646] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 559.400646] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 559.400646] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 559.423792] nova-conductor[54074]: DEBUG nova.quota [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Getting quotas for project 3eda153b02f14cdb99e65d35ebd0a260. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 559.434848] nova-conductor[54074]: DEBUG nova.quota [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Getting quotas for user 81fc47df516f43358054c51be332d998 and project 3eda153b02f14cdb99e65d35ebd0a260. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 559.441088] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] [instance: 9ebd0ef6-4a2d-414e-88e9-9b3cc739bb55] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 559.441614] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 559.441826] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 559.442266] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 559.472086] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] [instance: 9ebd0ef6-4a2d-414e-88e9-9b3cc739bb55] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 559.472848] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 559.473070] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 559.473256] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 559.615524] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 559.615771] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 559.615947] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1bf2d3ef-46ea-4fa4-8ca2-07de0503504d tempest-ServerDiagnosticsV248Test-1910275868 tempest-ServerDiagnosticsV248Test-1910275868-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 560.271846] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Expecting reply to msg b5643d2d3f724652ad42a6fc8d6db55d in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 560.466523] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] [instance: 3888f958-a6ea-4159-b2be-1b2b5781f41f] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port e90e5d3a-8fe5-43ff-b4d1-6a967cae6b5a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 3888f958-a6ea-4159-b2be-1b2b5781f41f was re-scheduled: Binding failed for port e90e5d3a-8fe5-43ff-b4d1-6a967cae6b5a, please check neutron logs for more information.\n'] [ 560.467112] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 560.467342] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3888f958-a6ea-4159-b2be-1b2b5781f41f.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3888f958-a6ea-4159-b2be-1b2b5781f41f. [ 560.467553] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] [instance: 3888f958-a6ea-4159-b2be-1b2b5781f41f] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3888f958-a6ea-4159-b2be-1b2b5781f41f. [ 560.581649] nova-conductor[54075]: DEBUG nova.network.neutron [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] [instance: 3888f958-a6ea-4159-b2be-1b2b5781f41f] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 560.687118] nova-conductor[54075]: DEBUG nova.network.neutron [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] [instance: 3888f958-a6ea-4159-b2be-1b2b5781f41f] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 560.694058] nova-conductor[54075]: DEBUG nova.network.neutron [None req-12f400c0-228e-4fe6-a20e-8d72e6d4c9e8 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] [instance: 3888f958-a6ea-4159-b2be-1b2b5781f41f] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 560.697240] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b5643d2d3f724652ad42a6fc8d6db55d [ 560.777873] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 560.814521] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 560.814759] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 560.814932] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 560.918102] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 560.918375] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 560.918540] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 560.918936] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 560.919076] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 560.919233] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 560.951600] nova-conductor[54075]: DEBUG nova.quota [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Getting quotas for project 8774e76528ec41a58ad05902aeb87360. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 560.960708] nova-conductor[54075]: DEBUG nova.quota [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Getting quotas for user bf0ea1832e344b5ebd8131b1aefea18e and project 8774e76528ec41a58ad05902aeb87360. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 560.969382] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] [instance: 5bde6403-4d82-4f8d-a3fb-fa4e75f0aaf7] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 560.969967] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 560.970289] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 560.970618] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 561.030386] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] [instance: 5bde6403-4d82-4f8d-a3fb-fa4e75f0aaf7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 561.031328] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 561.031328] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 561.031443] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 561.179614] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 561.179614] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 561.179614] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 564.141935] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Expecting reply to msg a2ea95cfd7c8446fb23f950514c85532 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 564.369076] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a2ea95cfd7c8446fb23f950514c85532 [ 564.650415] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 564.671145] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 564.671337] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 564.671337] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 564.739753] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 564.739982] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 564.740201] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 564.741889] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 564.742012] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 564.742179] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 564.752534] nova-conductor[54074]: DEBUG nova.quota [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Getting quotas for project b55538da1d3740f09e3c5ba6022ec17e. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 564.758584] nova-conductor[54074]: DEBUG nova.quota [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Getting quotas for user 809b3bdb71dc4e62a733190a8ad2e71b and project b55538da1d3740f09e3c5ba6022ec17e. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 564.765053] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] [instance: 2a473614-2051-47ab-a9bc-f87385a264cd] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 564.765707] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 564.765964] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 564.768208] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 564.771026] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] [instance: 2a473614-2051-47ab-a9bc-f87385a264cd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 564.771884] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 564.772247] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 564.772719] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 564.802349] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 564.802591] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 564.802846] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 564.826838] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] [instance: 834573d9-496f-4c80-b157-e1b12f799418] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 051993b2-9892-4974-a10f-35a4bdf58ec2, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 834573d9-496f-4c80-b157-e1b12f799418 was re-scheduled: Binding failed for port 051993b2-9892-4974-a10f-35a4bdf58ec2, please check neutron logs for more information.\n'] [ 564.827559] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 564.828272] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 834573d9-496f-4c80-b157-e1b12f799418.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 834573d9-496f-4c80-b157-e1b12f799418. [ 564.829197] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] [instance: 834573d9-496f-4c80-b157-e1b12f799418] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 834573d9-496f-4c80-b157-e1b12f799418. [ 564.893270] nova-conductor[54074]: DEBUG nova.network.neutron [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] [instance: 834573d9-496f-4c80-b157-e1b12f799418] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 565.695279] nova-conductor[54074]: DEBUG nova.network.neutron [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] [instance: 834573d9-496f-4c80-b157-e1b12f799418] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 565.703460] nova-conductor[54074]: DEBUG nova.network.neutron [None req-bbae6cd6-d8e2-4aa4-b815-9cd0580620e8 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] [instance: 834573d9-496f-4c80-b157-e1b12f799418] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 568.112205] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Expecting reply to msg 011afae42a934ce6b891e73503e0fe85 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 568.371940] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 011afae42a934ce6b891e73503e0fe85 [ 568.617240] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 568.634933] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 568.635304] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 568.635589] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 568.703517] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 568.704266] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 568.704653] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 568.705337] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 568.706005] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 568.706438] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 568.728733] nova-conductor[54074]: DEBUG nova.quota [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Getting quotas for project cacd2ccfb22c4f73abbe2d5670d1fbb9. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 568.733403] nova-conductor[54074]: DEBUG nova.quota [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Getting quotas for user 50add8117fb046ff9d5efdc0f40fcbfb and project cacd2ccfb22c4f73abbe2d5670d1fbb9. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 568.740747] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] [instance: 32a8ec11-f3ca-4df2-8231-3ce68c06bbcd] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 568.742207] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 568.742434] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 568.742604] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 568.757052] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] [instance: 32a8ec11-f3ca-4df2-8231-3ce68c06bbcd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 568.757052] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 568.757052] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 568.757052] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 568.781183] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 568.781565] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 568.781923] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 569.290919] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Expecting reply to msg e9c317510f0d4546bc7351c7d100bfc8 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 569.452716] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e9c317510f0d4546bc7351c7d100bfc8 [ 569.792905] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 569.816097] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 569.816624] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 569.816936] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 569.871578] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 569.871826] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 569.872026] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 569.872405] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 569.872571] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 569.873048] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 569.886203] nova-conductor[54074]: DEBUG nova.quota [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Getting quotas for project 423f777bec3c474a91970fce3e308097. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 569.890081] nova-conductor[54074]: DEBUG nova.quota [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Getting quotas for user e4ea22bb21004f69a2b27d306493db45 and project 423f777bec3c474a91970fce3e308097. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 569.898145] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: c742fcf9-ac27-4a04-81a2-d99741dba794] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 569.898857] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 569.899074] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 569.899243] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 569.905452] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: c742fcf9-ac27-4a04-81a2-d99741dba794] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 569.906265] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 569.906464] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 569.906629] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 569.943332] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 569.943512] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 569.943679] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 570.927793] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=3 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 571.034016] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=9,nova_cell1:SAVEPOINT=3,nova_cell1:INSERT=24,nova_cell1:RELEASE=3 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 571.128178] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=20,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 571.162701] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=16,nova_api:DELETE=2,nova_api:UPDATE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 572.737189] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Expecting reply to msg 1cfb8defa6e141d7b309572906e0a480 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 572.906519] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1cfb8defa6e141d7b309572906e0a480 [ 573.242287] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 573.260155] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 573.260468] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 573.260642] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 573.325713] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 573.326067] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 573.326335] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 573.326774] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 573.327035] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 573.327304] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 573.346076] nova-conductor[54075]: DEBUG nova.quota [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Getting quotas for project 88391b50bec241769f291a70ba99dfb9. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 573.362277] nova-conductor[54075]: DEBUG nova.quota [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Getting quotas for user 7677135908304a35935a9de31e0e5fd3 and project 88391b50bec241769f291a70ba99dfb9. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 573.373853] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] [instance: d24eec8f-565a-4a02-834c-267e633ebb12] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 573.374443] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 573.374693] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 573.374822] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 573.383429] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] [instance: d24eec8f-565a-4a02-834c-267e633ebb12] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 573.383656] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 573.383854] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 573.384030] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 573.410978] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 573.411261] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 573.411463] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 573.432063] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Expecting reply to msg f7c362b4c7f14572a81be74b3d88718d in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 573.601975] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f7c362b4c7f14572a81be74b3d88718d [ 573.944082] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 573.965258] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Expecting reply to msg c3c1cf1435df4b798bb927286faa1484 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 573.979288] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 573.979597] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 573.979773] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 574.112730] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 574.112964] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 574.113127] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 574.113494] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 574.114025] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 574.115214] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 574.135966] nova-conductor[54075]: DEBUG nova.quota [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Getting quotas for project 3c9199b442e5466390387d8884f8bc96. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 574.140873] nova-conductor[54075]: DEBUG nova.quota [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Getting quotas for user 8d76306b5db04e97a1c3e28ffdc73d78 and project 3c9199b442e5466390387d8884f8bc96. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 574.147372] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c3c1cf1435df4b798bb927286faa1484 [ 574.147808] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] [instance: 39901fc8-8fc8-4812-936e-0ded3811d61c] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 574.148672] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 574.148672] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 574.148880] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 574.166380] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] [instance: 39901fc8-8fc8-4812-936e-0ded3811d61c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 574.167144] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 574.167370] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 574.167534] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 574.209910] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 574.209910] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 574.209910] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 574.414318] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Expecting reply to msg 36373e5deec74b41a1509355c62f6415 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 574.472875] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 574.479013] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Expecting reply to msg f3ea02a4ba714ba389dd9b38f5929492 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 574.498715] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 574.498984] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 574.499210] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 574.580427] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 574.580680] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 574.580856] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 574.581509] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 574.581764] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 574.581945] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 574.601045] nova-conductor[54074]: DEBUG nova.quota [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Getting quotas for project b0e5d54143934f498a4a7849ec99a645. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 574.603646] nova-conductor[54074]: DEBUG nova.quota [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Getting quotas for user f67ad126d9354c9e9cd8ac79b829fb4a and project b0e5d54143934f498a4a7849ec99a645. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 574.611669] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] [instance: d91c5dae-4ece-4718-a16b-534729f7ba49] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 574.612279] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 574.612485] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 574.612648] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 574.613193] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 36373e5deec74b41a1509355c62f6415 [ 574.622469] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] [instance: d91c5dae-4ece-4718-a16b-534729f7ba49] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 574.623916] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 574.624387] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 574.624567] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 574.642392] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f3ea02a4ba714ba389dd9b38f5929492 [ 574.656991] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 574.657227] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 574.657396] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 574.921163] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 574.937711] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 574.937970] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 574.938149] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 574.975165] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 574.975739] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 574.975983] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 574.976594] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 574.976818] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 574.977124] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 574.983100] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 574.989565] nova-conductor[54075]: DEBUG nova.quota [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Getting quotas for project 39bd2e59c7e344c2922dbc693a6e0f7c. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 574.991727] nova-conductor[54075]: DEBUG nova.quota [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Getting quotas for user 77a7bbc94ac845e0b986f36581fffd10 and project 39bd2e59c7e344c2922dbc693a6e0f7c. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 574.996887] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] [instance: 2fb7c1e4-d756-4528-914e-b924c5a3be38] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 574.997335] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 574.997654] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 574.997722] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 574.999677] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 574.999936] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 575.000194] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 575.002661] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] [instance: 2fb7c1e4-d756-4528-914e-b924c5a3be38] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 575.003327] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 575.003548] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 575.003701] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 575.058867] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 575.059138] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 575.059307] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 575.059765] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 575.059885] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 575.060062] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 575.062901] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 575.063177] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 575.063288] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 575.078770] nova-conductor[54074]: DEBUG nova.quota [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Getting quotas for project d5eead97e11a4688b0058d50d5bef6ef. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 575.086506] nova-conductor[54074]: DEBUG nova.quota [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Getting quotas for user 332259bb7d2841e385dc46a558da2fd9 and project d5eead97e11a4688b0058d50d5bef6ef. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 575.092757] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] [instance: f1239cdd-d1b3-4494-8204-0fe150737579] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 575.093240] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 575.093553] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 575.093787] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 575.101557] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] [instance: f1239cdd-d1b3-4494-8204-0fe150737579] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 575.102375] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 575.102585] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 575.102793] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 575.155115] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 575.155345] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 575.155520] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 576.005285] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Expecting reply to msg 14b314c690a7462b9c110dfbad238776 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 576.116624] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Expecting reply to msg f00ac0716e6946ae826e4c606f612722 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 576.208069] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 14b314c690a7462b9c110dfbad238776 [ 576.281484] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f00ac0716e6946ae826e4c606f612722 [ 576.508974] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 576.525321] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 576.525534] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 576.525706] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 576.608505] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 576.608704] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 576.608869] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 576.609323] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 576.609836] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 576.609836] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 576.619022] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 576.633879] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 576.634107] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 576.634286] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 576.638212] nova-conductor[54074]: DEBUG nova.quota [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Getting quotas for project a65a1ee33b854b9c94073bc5e1316ebe. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 576.648660] nova-conductor[54074]: DEBUG nova.quota [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Getting quotas for user 73751885e6db46f99b73d3ca8786f135 and project a65a1ee33b854b9c94073bc5e1316ebe. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 576.658578] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] [instance: ce54ba3d-2cd5-4400-b334-8443ef73bbff] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 576.659361] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 576.659877] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 576.660231] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 576.663812] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] [instance: ce54ba3d-2cd5-4400-b334-8443ef73bbff] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 576.666809] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 576.667132] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 576.667308] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 576.693019] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 576.693019] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 576.693019] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 576.693310] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 576.694146] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 576.694347] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 576.713435] nova-conductor[54075]: DEBUG nova.quota [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Getting quotas for project 003b76634a314c8ba271d8ff8a84f4cb. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 576.721515] nova-conductor[54075]: DEBUG nova.quota [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Getting quotas for user 977af06e03fb4180b6c5cfd5ae8aa0cf and project 003b76634a314c8ba271d8ff8a84f4cb. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 576.738973] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] [instance: 5f4a5c62-85f1-47ee-b702-1785bfe62f48] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 576.739704] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 576.739785] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 576.739910] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 576.742138] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 576.742824] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 576.742824] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 576.752210] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] [instance: 5f4a5c62-85f1-47ee-b702-1785bfe62f48] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 576.752210] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 576.752210] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 576.752210] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 576.798195] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 576.798424] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 576.798599] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 577.510644] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Expecting reply to msg 6a9cf741ab5f410ba78ffa04a66b82cb in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 577.654920] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6a9cf741ab5f410ba78ffa04a66b82cb [ 578.012901] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 578.027322] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 578.028562] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 578.029368] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 578.076356] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 578.076885] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 578.077126] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 578.077576] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 578.077861] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 578.078031] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 578.090464] nova-conductor[54075]: DEBUG nova.quota [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Getting quotas for project 4a15f084eff34d2b840889a457bdaeb3. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 578.093474] nova-conductor[54075]: DEBUG nova.quota [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Getting quotas for user 23ccfeaefb6445009a1e73e9e8c5d07b and project 4a15f084eff34d2b840889a457bdaeb3. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 578.102996] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] [instance: 56f9bb28-2770-46aa-9d95-f60cdeae0967] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 578.103990] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 578.104625] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 578.105002] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 578.111703] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] [instance: 56f9bb28-2770-46aa-9d95-f60cdeae0967] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 578.112582] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 578.112793] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 578.112960] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 578.132782] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 578.133061] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 578.133260] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 580.012309] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] [instance: 9cbf81d9-92b5-4edf-8e7e-f4caa8f44f66] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port bcd13e48-db65-4575-a819-060b93efb820, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 9cbf81d9-92b5-4edf-8e7e-f4caa8f44f66 was re-scheduled: Binding failed for port bcd13e48-db65-4575-a819-060b93efb820, please check neutron logs for more information.\n'] [ 580.012869] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 580.013529] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9cbf81d9-92b5-4edf-8e7e-f4caa8f44f66.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9cbf81d9-92b5-4edf-8e7e-f4caa8f44f66. [ 580.013923] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] [instance: 9cbf81d9-92b5-4edf-8e7e-f4caa8f44f66] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9cbf81d9-92b5-4edf-8e7e-f4caa8f44f66. [ 580.043567] nova-conductor[54074]: DEBUG nova.network.neutron [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] [instance: 9cbf81d9-92b5-4edf-8e7e-f4caa8f44f66] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 580.080558] nova-conductor[54074]: DEBUG nova.network.neutron [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] [instance: 9cbf81d9-92b5-4edf-8e7e-f4caa8f44f66] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 580.084396] nova-conductor[54074]: DEBUG nova.network.neutron [None req-4797d1cc-2262-489e-896f-329d69801242 tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] [instance: 9cbf81d9-92b5-4edf-8e7e-f4caa8f44f66] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 580.295085] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Expecting reply to msg 30ad306c317e49d591e5ba648cc7ed59 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 580.487879] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 30ad306c317e49d591e5ba648cc7ed59 [ 580.798111] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 580.813012] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 580.813249] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 580.813439] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 580.857785] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 580.858114] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 580.858311] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 580.858664] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 580.858852] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 580.859024] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 580.867536] nova-conductor[54075]: DEBUG nova.quota [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Getting quotas for project 6ebd77f41f294e62bf81eacf4e449d63. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 580.869622] nova-conductor[54075]: DEBUG nova.quota [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Getting quotas for user 5fe43f98ba574c19847bad2d5ed221c7 and project 6ebd77f41f294e62bf81eacf4e449d63. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 580.874491] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] [instance: 35411b03-ace3-40da-8c3e-3872ac003bd3] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 580.875069] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 580.875326] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 580.875434] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 580.878277] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] [instance: 35411b03-ace3-40da-8c3e-3872ac003bd3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 580.878967] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 580.879134] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 580.879300] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 580.896977] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 580.897154] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 580.897315] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 583.281100] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=306,nova_cell1:UPDATE=143,nova_cell1:SAVEPOINT=22,nova_cell1:RELEASE=22,nova_cell1:INSERT=23 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 586.613061] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=13 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 586.669239] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=37,nova_cell1:SAVEPOINT=9,nova_cell1:INSERT=93,nova_cell1:RELEASE=9 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 586.697085] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=74,nova_api:UPDATE=7,nova_api:DELETE=7 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 586.702290] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=69,nova_api:UPDATE=5,nova_api:DELETE=5 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 587.186192] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=318,nova_cell1:UPDATE=140,nova_cell1:INSERT=20,nova_cell1:SAVEPOINT=21,nova_cell1:RELEASE=21 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 588.129051] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Expecting reply to msg 0d2d4e0fc2ce4dd5b3263807943951b9 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 588.130356] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Expecting reply to msg d890d27d67954144adffe1b16370c7ef in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 588.302401] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d890d27d67954144adffe1b16370c7ef [ 588.328889] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0d2d4e0fc2ce4dd5b3263807943951b9 [ 588.631918] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 588.634275] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 588.648554] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.648672] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.648872] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.648999] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.649049] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 588.655489] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 588.702720] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.702720] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.702855] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 588.703223] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.703413] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.704972] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 588.705683] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.706708] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.707432] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 588.708279] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.709793] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.710216] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 588.722046] nova-conductor[54074]: DEBUG nova.quota [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Getting quotas for project f74de90347d44395b052d75738d6a065. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 588.726100] nova-conductor[54074]: DEBUG nova.quota [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Getting quotas for user 2857ea8202fc46b09da06c4ac904df95 and project f74de90347d44395b052d75738d6a065. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 588.732297] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] [instance: fa25ef8c-cb4a-4c5b-8157-3d2e1a2e3932] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 588.733009] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.733425] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.733777] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 588.736438] nova-conductor[54075]: DEBUG nova.quota [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Getting quotas for project 6b7ae63045184f9999aa13025fdc6b26. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 588.739635] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] [instance: fa25ef8c-cb4a-4c5b-8157-3d2e1a2e3932] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 588.740687] nova-conductor[54075]: DEBUG nova.quota [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Getting quotas for user 7ca9c1043bfc469e8230e5c6bfc5f48a and project 6b7ae63045184f9999aa13025fdc6b26. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 588.741188] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.741741] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.743079] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 588.753421] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] [instance: 3454f8a4-a9c7-4622-9cdc-36f683b6f3ee] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 588.756722] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.756722] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.756722] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 588.757482] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] [instance: 3454f8a4-a9c7-4622-9cdc-36f683b6f3ee] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 588.758231] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.758362] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.758525] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 588.791115] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.791614] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.791915] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.792154] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 588.792657] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.792852] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 589.339401] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Expecting reply to msg ec151209f9704d0ea538b14c43ff47bd in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 589.432673] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 22179940-4e5b-4879-be19-a9addb0a628c] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 10ebfd79-b01d-46c0-8fcf-e500ebe6ca84, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 22179940-4e5b-4879-be19-a9addb0a628c was re-scheduled: Binding failed for port 10ebfd79-b01d-46c0-8fcf-e500ebe6ca84, please check neutron logs for more information.\n'] [ 589.433222] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 589.433534] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 22179940-4e5b-4879-be19-a9addb0a628c.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 22179940-4e5b-4879-be19-a9addb0a628c. [ 589.433803] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 22179940-4e5b-4879-be19-a9addb0a628c] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 22179940-4e5b-4879-be19-a9addb0a628c. [ 589.467349] nova-conductor[54074]: DEBUG nova.network.neutron [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 22179940-4e5b-4879-be19-a9addb0a628c] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 589.508693] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ec151209f9704d0ea538b14c43ff47bd [ 589.512701] nova-conductor[54074]: DEBUG nova.network.neutron [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 22179940-4e5b-4879-be19-a9addb0a628c] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 589.516214] nova-conductor[54074]: DEBUG nova.network.neutron [None req-640de2c7-753c-4d78-b3be-95d920cb7fe3 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 22179940-4e5b-4879-be19-a9addb0a628c] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 589.842716] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 589.860437] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 589.860739] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 589.860842] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 589.912889] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 589.913481] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 589.913481] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 589.913672] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 589.913854] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 589.914028] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 589.930373] nova-conductor[54075]: DEBUG nova.quota [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Getting quotas for project 1334c2de495a461fa55e937653441f68. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 589.945094] nova-conductor[54075]: DEBUG nova.quota [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Getting quotas for user 13438e8357694da8b1843f8d44270eac and project 1334c2de495a461fa55e937653441f68. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 589.972698] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] [instance: 80a15f60-4843-4a59-a6c1-0d5624609672] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 589.973321] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 589.973580] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 589.973782] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 589.988583] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] [instance: 80a15f60-4843-4a59-a6c1-0d5624609672] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 589.991280] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 589.991697] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 589.991764] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 590.020070] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 590.020530] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 590.020961] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 590.973395] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] [instance: 15dc4e5a-da5b-4657-8aec-f501d35d7a58] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 38c57439-2b4f-48ef-884c-f248749447a4, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 15dc4e5a-da5b-4657-8aec-f501d35d7a58 was re-scheduled: Binding failed for port 38c57439-2b4f-48ef-884c-f248749447a4, please check neutron logs for more information.\n'] [ 590.973779] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 590.973902] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 15dc4e5a-da5b-4657-8aec-f501d35d7a58.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 15dc4e5a-da5b-4657-8aec-f501d35d7a58. [ 590.974213] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] [instance: 15dc4e5a-da5b-4657-8aec-f501d35d7a58] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 15dc4e5a-da5b-4657-8aec-f501d35d7a58. [ 591.009247] nova-conductor[54075]: DEBUG nova.network.neutron [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] [instance: 15dc4e5a-da5b-4657-8aec-f501d35d7a58] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 591.060201] nova-conductor[54075]: DEBUG nova.network.neutron [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] [instance: 15dc4e5a-da5b-4657-8aec-f501d35d7a58] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 591.062775] nova-conductor[54075]: DEBUG nova.network.neutron [None req-2936a47b-0996-4d83-bf82-1b2c3804d4cd tempest-ServersAdminTestJSON-1198871778 tempest-ServersAdminTestJSON-1198871778-project-member] [instance: 15dc4e5a-da5b-4657-8aec-f501d35d7a58] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 592.487569] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Expecting reply to msg 3596c3d2ce754d70b0fb652af4bc4f47 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 592.659562] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3596c3d2ce754d70b0fb652af4bc4f47 [ 593.002853] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Took 0.52 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 593.016768] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 593.016995] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 593.017161] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 593.092691] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 593.093482] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 593.093872] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 593.094196] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 593.094396] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 593.094805] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 593.105988] nova-conductor[54075]: DEBUG nova.quota [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Getting quotas for project c19034722a8742e7b62e6107e9028a9c. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 593.110049] nova-conductor[54075]: DEBUG nova.quota [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Getting quotas for user 1bc490e03afb422a86e04901d36d6945 and project c19034722a8742e7b62e6107e9028a9c. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 593.116120] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] [instance: 15e2e743-070f-4545-b976-ced38fd99198] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 593.116700] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 593.117008] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 593.117241] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 593.124338] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] [instance: 15e2e743-070f-4545-b976-ced38fd99198] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 593.125254] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 593.125565] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 593.125921] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 593.162352] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 593.162531] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 593.163066] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 593.617784] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Expecting reply to msg ca3a9e47272c4853a6427ba002a0c2f9 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 593.795977] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ca3a9e47272c4853a6427ba002a0c2f9 [ 594.120594] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 594.138603] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 594.139008] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 594.139087] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 594.201199] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 594.201504] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 594.201730] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 594.202182] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 594.202614] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 594.202614] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 594.219910] nova-conductor[54075]: DEBUG nova.quota [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Getting quotas for project f0ff66f09fe24c15a6498a601ccb43d5. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 594.223245] nova-conductor[54075]: DEBUG nova.quota [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Getting quotas for user bedc426b7fca4be7bb9c9d2edd41bc83 and project f0ff66f09fe24c15a6498a601ccb43d5. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 594.229369] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] [instance: 2fa640c2-b433-4581-be4b-0673c1451043] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 594.230660] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 594.230660] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 594.230660] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 594.234747] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] [instance: 2fa640c2-b433-4581-be4b-0673c1451043] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 594.235406] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 594.235608] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 594.235772] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 594.266243] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 594.266485] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 594.266671] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 594.811738] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Expecting reply to msg 3faf46f5816048d1a12b84c561be9c43 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 594.983112] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3faf46f5816048d1a12b84c561be9c43 [ 595.312894] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 595.348556] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 595.348556] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 595.348556] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 595.425285] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 595.425425] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 595.425596] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 595.425984] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 595.426168] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 595.426325] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 595.441127] nova-conductor[54074]: DEBUG nova.quota [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Getting quotas for project 61a866168186462d9d849072a1ff25f1. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 595.445044] nova-conductor[54074]: DEBUG nova.quota [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Getting quotas for user 6349e1aff7d945a6a471b1f4e826b23f and project 61a866168186462d9d849072a1ff25f1. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 595.451087] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 535045d9-108e-4e88-82f0-9da98f2f55a6] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 595.451254] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 595.451511] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 595.451649] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 595.461194] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 535045d9-108e-4e88-82f0-9da98f2f55a6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 595.461994] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 595.462229] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 595.462410] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 595.491859] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 595.492081] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 595.492265] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 595.706532] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] [instance: 5bde6403-4d82-4f8d-a3fb-fa4e75f0aaf7] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 228e8a13-db93-4bfa-b8bc-ab5292e785b1, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 5bde6403-4d82-4f8d-a3fb-fa4e75f0aaf7 was re-scheduled: Binding failed for port 228e8a13-db93-4bfa-b8bc-ab5292e785b1, please check neutron logs for more information.\n'] [ 595.707200] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 595.707430] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5bde6403-4d82-4f8d-a3fb-fa4e75f0aaf7.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5bde6403-4d82-4f8d-a3fb-fa4e75f0aaf7. [ 595.707747] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] [instance: 5bde6403-4d82-4f8d-a3fb-fa4e75f0aaf7] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5bde6403-4d82-4f8d-a3fb-fa4e75f0aaf7. [ 595.750335] nova-conductor[54075]: DEBUG nova.network.neutron [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] [instance: 5bde6403-4d82-4f8d-a3fb-fa4e75f0aaf7] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 595.808852] nova-conductor[54075]: DEBUG nova.network.neutron [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] [instance: 5bde6403-4d82-4f8d-a3fb-fa4e75f0aaf7] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 595.812577] nova-conductor[54075]: DEBUG nova.network.neutron [None req-d4e425a3-dfe5-4f27-b4c2-e8ecf97207b5 tempest-VolumesAssistedSnapshotsTest-857465433 tempest-VolumesAssistedSnapshotsTest-857465433-project-member] [instance: 5bde6403-4d82-4f8d-a3fb-fa4e75f0aaf7] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 597.491905] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] [instance: 2a473614-2051-47ab-a9bc-f87385a264cd] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port ebefe3f7-1348-4203-ab33-a289a4fa30de, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 2a473614-2051-47ab-a9bc-f87385a264cd was re-scheduled: Binding failed for port ebefe3f7-1348-4203-ab33-a289a4fa30de, please check neutron logs for more information.\n'] [ 597.494781] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 597.494953] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2a473614-2051-47ab-a9bc-f87385a264cd.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2a473614-2051-47ab-a9bc-f87385a264cd. [ 597.495179] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] [instance: 2a473614-2051-47ab-a9bc-f87385a264cd] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2a473614-2051-47ab-a9bc-f87385a264cd. [ 597.536402] nova-conductor[54075]: DEBUG nova.network.neutron [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] [instance: 2a473614-2051-47ab-a9bc-f87385a264cd] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 597.567152] nova-conductor[54075]: DEBUG nova.network.neutron [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] [instance: 2a473614-2051-47ab-a9bc-f87385a264cd] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 597.571956] nova-conductor[54075]: DEBUG nova.network.neutron [None req-48072dcd-56b4-4ba7-9d78-0fba124fc2c3 tempest-ServersWithSpecificFlavorTestJSON-787125351 tempest-ServersWithSpecificFlavorTestJSON-787125351-project-member] [instance: 2a473614-2051-47ab-a9bc-f87385a264cd] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 597.728420] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 597.728420] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 597.728420] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 597.760029] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Expecting reply to msg 92ce17ce70b64eae852cad1638425f72 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 597.920281] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Expecting reply to msg 815fa72f913846558f89128dc8515645 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 597.937975] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 92ce17ce70b64eae852cad1638425f72 [ 598.050406] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 815fa72f913846558f89128dc8515645 [ 598.263095] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 598.285526] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.285859] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.286133] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.341030] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.341030] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.341268] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.341448] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.341657] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.341817] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.357481] nova-conductor[54074]: DEBUG nova.quota [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Getting quotas for project e587eb843ad3412dbd94b5fde13771b7. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 598.361549] nova-conductor[54074]: DEBUG nova.quota [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Getting quotas for user 98ec2a1e4163442a9dfd7ce25bf13d6f and project e587eb843ad3412dbd94b5fde13771b7. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 598.381290] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] [instance: 83a25ff9-cc7d-4917-95cc-e621884bcee8] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 598.381900] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.382164] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.382409] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.393122] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] [instance: 83a25ff9-cc7d-4917-95cc-e621884bcee8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 598.393895] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.394131] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.394352] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.427886] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 598.433747] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.434421] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.434787] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.446454] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.446703] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.446876] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.489795] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.490024] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.490208] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.490784] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.490784] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.490986] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.499529] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Expecting reply to msg 5f9d8c6934a942b6a3acb79a6ba88a11 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 598.520775] nova-conductor[54075]: DEBUG nova.quota [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Getting quotas for project f0564c0961c84ae29d228b3b98cc113a. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 598.541720] nova-conductor[54075]: DEBUG nova.quota [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Getting quotas for user 13ba568bd2274469b24604d938cd3e9d and project f0564c0961c84ae29d228b3b98cc113a. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 598.584342] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] [instance: b537150e-9136-4fa4-b092-4f4995b918b7] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 598.584807] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.585480] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.585663] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.594046] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] [instance: b537150e-9136-4fa4-b092-4f4995b918b7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 598.594890] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.595143] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.595320] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.617329] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.617585] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.617753] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.660735] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5f9d8c6934a942b6a3acb79a6ba88a11 [ 599.002509] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 599.024416] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 599.024905] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 599.024905] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 599.087961] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 599.088501] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 599.088829] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 599.089374] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 599.089667] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 599.090067] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 599.100742] nova-conductor[54074]: DEBUG nova.quota [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Getting quotas for project a3d972fbe68a485ea9884936e907fae7. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 599.105434] nova-conductor[54074]: DEBUG nova.quota [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Getting quotas for user fe2d09b691d94a748f19caaf87f70e9e and project a3d972fbe68a485ea9884936e907fae7. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 599.111094] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] [instance: fe9756ba-0eb5-41ad-913f-e933f97542cb] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 599.111664] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 599.111944] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 599.112145] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 599.117430] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] [instance: fe9756ba-0eb5-41ad-913f-e933f97542cb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 599.118131] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 599.118341] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 599.118508] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 599.153322] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 599.153576] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 599.153757] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 600.767844] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Expecting reply to msg b3f8aa5ea8384c2c916ed838647941c7 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 600.977924] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b3f8aa5ea8384c2c916ed838647941c7 [ 601.270305] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 601.284791] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 601.284883] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 601.285053] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 601.345116] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 601.345441] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 601.345634] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 601.346084] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 601.346273] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 601.346427] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 601.355162] nova-conductor[54074]: DEBUG nova.quota [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Getting quotas for project 4678fb5a620741c1b27309ff6f4597eb. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 601.358498] nova-conductor[54074]: DEBUG nova.quota [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Getting quotas for user 69c1f9cac1b84ea18a4065e389324d97 and project 4678fb5a620741c1b27309ff6f4597eb. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 601.368153] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] [instance: 11a6eaa1-0d35-49cf-9341-b74129cf087b] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 601.368153] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 601.368153] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 601.368153] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 601.374918] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] [instance: 11a6eaa1-0d35-49cf-9341-b74129cf087b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 601.375585] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 601.375793] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 601.376218] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 601.391990] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 601.392243] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 601.392415] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 601.748481] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Expecting reply to msg 1fe75d1aa9b748fbbfffc47d71dbe59c in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 601.921474] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1fe75d1aa9b748fbbfffc47d71dbe59c [ 602.251024] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 602.287028] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 602.287717] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 602.287825] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 602.390844] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 602.391181] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 602.391481] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 602.392224] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 602.392565] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 602.392778] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 602.436247] nova-conductor[54075]: DEBUG nova.quota [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Getting quotas for project 21cdf8675fb347c2874d912dcb8ac002. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 602.461555] nova-conductor[54075]: DEBUG nova.quota [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Getting quotas for user 23f9946393284244aca0c29201c37fa4 and project 21cdf8675fb347c2874d912dcb8ac002. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 602.483702] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 3a4e1dcc-610f-4037-94e9-c9815c12ed1d] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 602.484456] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 602.484658] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 602.484890] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 602.505967] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 3a4e1dcc-610f-4037-94e9-c9815c12ed1d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 602.507279] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 602.507279] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 602.507622] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 602.554727] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 602.555007] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 602.555235] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 606.515666] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] [instance: 32a8ec11-f3ca-4df2-8231-3ce68c06bbcd] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 636f6a96-849f-411e-a602-2d048aa9867a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 32a8ec11-f3ca-4df2-8231-3ce68c06bbcd was re-scheduled: Binding failed for port 636f6a96-849f-411e-a602-2d048aa9867a, please check neutron logs for more information.\n'] [ 606.516081] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 606.516833] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 32a8ec11-f3ca-4df2-8231-3ce68c06bbcd.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 32a8ec11-f3ca-4df2-8231-3ce68c06bbcd. [ 606.516833] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] [instance: 32a8ec11-f3ca-4df2-8231-3ce68c06bbcd] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 32a8ec11-f3ca-4df2-8231-3ce68c06bbcd. [ 606.545941] nova-conductor[54075]: DEBUG nova.network.neutron [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] [instance: 32a8ec11-f3ca-4df2-8231-3ce68c06bbcd] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 606.572053] nova-conductor[54075]: DEBUG nova.network.neutron [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] [instance: 32a8ec11-f3ca-4df2-8231-3ce68c06bbcd] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 606.581874] nova-conductor[54075]: DEBUG nova.network.neutron [None req-53e75428-6f73-420a-ac14-a07867f8a23c tempest-ServersAdminNegativeTestJSON-53709098 tempest-ServersAdminNegativeTestJSON-53709098-project-member] [instance: 32a8ec11-f3ca-4df2-8231-3ce68c06bbcd] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 607.554670] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Expecting reply to msg 8d72e439700548159ba4e53ea027687d in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 607.689158] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8d72e439700548159ba4e53ea027687d [ 608.058972] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 608.082303] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 608.083039] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 608.083039] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 608.147394] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 608.147394] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 608.147394] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 608.147394] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 608.147394] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 608.147394] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 608.157273] nova-conductor[54075]: DEBUG nova.quota [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Getting quotas for project aa59611b27bb41beb282e68ccfa6fadc. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 608.160818] nova-conductor[54075]: DEBUG nova.quota [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Getting quotas for user ba5969be1a254281b4dffd81aa84ae7e and project aa59611b27bb41beb282e68ccfa6fadc. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 608.166397] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 0e018d70-d6dd-4f79-bb03-14b815645562] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 608.166991] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 608.167372] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 608.167680] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 608.172873] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 0e018d70-d6dd-4f79-bb03-14b815645562] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 608.178304] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 608.178304] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 608.178304] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 608.204192] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 608.204192] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 608.204192] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 608.498976] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: c742fcf9-ac27-4a04-81a2-d99741dba794] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 4bf704f8-170e-48fa-9d73-18fa03b4afe7, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance c742fcf9-ac27-4a04-81a2-d99741dba794 was re-scheduled: Binding failed for port 4bf704f8-170e-48fa-9d73-18fa03b4afe7, please check neutron logs for more information.\n'] [ 608.499815] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 608.506478] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c742fcf9-ac27-4a04-81a2-d99741dba794.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c742fcf9-ac27-4a04-81a2-d99741dba794. [ 608.507157] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: c742fcf9-ac27-4a04-81a2-d99741dba794] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c742fcf9-ac27-4a04-81a2-d99741dba794. [ 608.544511] nova-conductor[54075]: DEBUG nova.network.neutron [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: c742fcf9-ac27-4a04-81a2-d99741dba794] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 608.599723] nova-conductor[54075]: DEBUG nova.network.neutron [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: c742fcf9-ac27-4a04-81a2-d99741dba794] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 608.606878] nova-conductor[54075]: DEBUG nova.network.neutron [None req-6dd56597-da9d-4a0f-a10b-4aa717800955 tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: c742fcf9-ac27-4a04-81a2-d99741dba794] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 609.287694] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Expecting reply to msg 915a003975104684bf0b1371f92c8616 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 609.464539] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 915a003975104684bf0b1371f92c8616 [ 609.795542] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 609.823919] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.824189] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.824365] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 609.881956] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.882627] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.882943] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 609.883508] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.883846] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.884167] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 609.907572] nova-conductor[54074]: DEBUG nova.quota [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Getting quotas for project 03e845d8aa4a47b6954ddf78c83c41aa. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 609.919832] nova-conductor[54074]: DEBUG nova.quota [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Getting quotas for user f9b5da32e11744ada3e612ea411872f9 and project 03e845d8aa4a47b6954ddf78c83c41aa. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 609.929489] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] [instance: c9b2ced5-a77c-4bff-b115-ce5c523be630] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 609.930087] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.930343] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.930546] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 609.936372] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] [instance: c9b2ced5-a77c-4bff-b115-ce5c523be630] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 609.937056] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.937267] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.937757] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 609.964154] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.964401] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.964610] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 610.348919] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] [instance: d24eec8f-565a-4a02-834c-267e633ebb12] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 5cc66433-8c1c-4902-a2f5-0d2c2f4a5ec1, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance d24eec8f-565a-4a02-834c-267e633ebb12 was re-scheduled: Binding failed for port 5cc66433-8c1c-4902-a2f5-0d2c2f4a5ec1, please check neutron logs for more information.\n'] [ 610.349563] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 610.349825] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d24eec8f-565a-4a02-834c-267e633ebb12.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d24eec8f-565a-4a02-834c-267e633ebb12. [ 610.355248] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] [instance: d24eec8f-565a-4a02-834c-267e633ebb12] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d24eec8f-565a-4a02-834c-267e633ebb12. [ 610.381676] nova-conductor[54074]: DEBUG nova.network.neutron [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] [instance: d24eec8f-565a-4a02-834c-267e633ebb12] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 610.438189] nova-conductor[54074]: DEBUG nova.network.neutron [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] [instance: d24eec8f-565a-4a02-834c-267e633ebb12] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 610.452929] nova-conductor[54074]: DEBUG nova.network.neutron [None req-fd717bfe-f2b1-46c2-b84c-d116f213f015 tempest-FloatingIPsAssociationTestJSON-143793456 tempest-FloatingIPsAssociationTestJSON-143793456-project-member] [instance: d24eec8f-565a-4a02-834c-267e633ebb12] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 611.279095] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Expecting reply to msg e25e64de9f4946a0aafb5d0093f75004 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 611.537417] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e25e64de9f4946a0aafb5d0093f75004 [ 611.786115] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 611.802566] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 611.802669] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 611.802773] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 611.901449] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 611.902295] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 611.902720] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 611.902955] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 611.903194] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 611.903395] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 611.919643] nova-conductor[54074]: DEBUG nova.quota [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Getting quotas for project 03e845d8aa4a47b6954ddf78c83c41aa. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 611.928360] nova-conductor[54074]: DEBUG nova.quota [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Getting quotas for user f9b5da32e11744ada3e612ea411872f9 and project 03e845d8aa4a47b6954ddf78c83c41aa. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 611.936130] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] [instance: 26a287d7-4602-4d83-8828-41870a49c343] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 611.936130] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 611.936130] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 611.936130] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 611.949504] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] [instance: 26a287d7-4602-4d83-8828-41870a49c343] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 611.950271] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 611.950486] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 611.950673] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 611.983620] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 611.983620] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 611.983620] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 613.363115] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Expecting reply to msg 0fb691eb3114494ea828a00ca91afc57 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 613.507163] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0fb691eb3114494ea828a00ca91afc57 [ 613.865178] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 613.880039] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 613.880039] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 613.880039] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 613.921005] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 613.921247] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 613.924311] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 613.924311] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 613.924311] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 613.924311] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 613.932997] nova-conductor[54075]: DEBUG nova.quota [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Getting quotas for project 801256c0ba804c308bafbf041533b989. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 613.936913] nova-conductor[54075]: DEBUG nova.quota [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Getting quotas for user 1e2a4ff94deb42b4a0dbb88b48c1960f and project 801256c0ba804c308bafbf041533b989. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 613.947925] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] [instance: dcdb35dd-3c01-4ba6-9a94-9ef3d49caa09] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 613.948472] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 613.948722] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 613.948935] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 613.959041] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] [instance: dcdb35dd-3c01-4ba6-9a94-9ef3d49caa09] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 613.959041] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 613.959041] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 613.959675] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 613.982387] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 613.982676] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 613.983166] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 614.048947] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Expecting reply to msg 07f46a3cc0cd43b09e0afb0f22c0afb4 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 614.250161] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 07f46a3cc0cd43b09e0afb0f22c0afb4 [ 614.548143] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 614.561168] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 614.561388] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 614.561750] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 614.596481] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 614.596711] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 614.596891] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 614.597256] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 614.597443] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 614.597604] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 614.613503] nova-conductor[54074]: DEBUG nova.quota [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Getting quotas for project 214dea28f048427e8a9eb82bcc2cb063. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 614.616940] nova-conductor[54074]: DEBUG nova.quota [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Getting quotas for user 8fa210afa0ba4ce8a50b24f78a4c69f8 and project 214dea28f048427e8a9eb82bcc2cb063. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 614.631388] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] [instance: 29715a53-7a71-4708-b522-e678fe5bd6a9] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 614.631928] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 614.632172] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 614.632346] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 614.636862] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] [instance: 29715a53-7a71-4708-b522-e678fe5bd6a9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 614.637517] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 614.637722] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 614.637954] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 614.653954] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 614.654166] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 614.654332] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 614.972693] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Expecting reply to msg deae6303f43342bc92f45f63ae4167c2 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 615.155345] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg deae6303f43342bc92f45f63ae4167c2 [ 615.475767] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 615.492801] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 615.493008] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 615.493178] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 615.542321] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 615.542856] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 615.543082] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 615.543500] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 615.543737] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 615.543945] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 615.557565] nova-conductor[54075]: DEBUG nova.quota [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Getting quotas for project 423f777bec3c474a91970fce3e308097. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 615.562602] nova-conductor[54075]: DEBUG nova.quota [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Getting quotas for user e4ea22bb21004f69a2b27d306493db45 and project 423f777bec3c474a91970fce3e308097. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 615.568335] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 446bd52c-4ffb-4e77-89fb-3e8535ceb4af] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 615.569022] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 615.569284] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 615.569642] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 615.580160] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 446bd52c-4ffb-4e77-89fb-3e8535ceb4af] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 615.581154] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 615.581375] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 615.581733] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 615.603967] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 615.604277] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 615.604576] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 616.813579] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Expecting reply to msg 0ef3ea0e747848a5940181ce10d1cc79 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 617.015715] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0ef3ea0e747848a5940181ce10d1cc79 [ 617.318153] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] [instance: 39901fc8-8fc8-4812-936e-0ded3811d61c] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port c9378bce-31f2-4b1a-a99e-35f01300a701, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 39901fc8-8fc8-4812-936e-0ded3811d61c was re-scheduled: Binding failed for port c9378bce-31f2-4b1a-a99e-35f01300a701, please check neutron logs for more information.\n'] [ 617.318589] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 617.318905] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 39901fc8-8fc8-4812-936e-0ded3811d61c.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 39901fc8-8fc8-4812-936e-0ded3811d61c. [ 617.319295] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] [instance: 39901fc8-8fc8-4812-936e-0ded3811d61c] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 39901fc8-8fc8-4812-936e-0ded3811d61c. [ 617.321619] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 617.368936] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 617.369233] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 617.369410] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 617.378452] nova-conductor[54075]: DEBUG nova.network.neutron [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] [instance: 39901fc8-8fc8-4812-936e-0ded3811d61c] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 617.396561] nova-conductor[54075]: DEBUG nova.network.neutron [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] [instance: 39901fc8-8fc8-4812-936e-0ded3811d61c] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 617.404977] nova-conductor[54075]: DEBUG nova.network.neutron [None req-e0393855-6a43-4693-ac68-c8f9bbb990a2 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993 tempest-FloatingIPsAssociationNegativeTestJSON-1419858993-project-member] [instance: 39901fc8-8fc8-4812-936e-0ded3811d61c] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 617.457532] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 617.457771] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 617.458190] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 617.458599] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 617.458785] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 617.458940] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 617.469623] nova-conductor[54075]: DEBUG nova.quota [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Getting quotas for project ae1d64a1d7b54a819ffa1c913bdcecaa. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 617.472829] nova-conductor[54075]: DEBUG nova.quota [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Getting quotas for user f6cf6e0db3914555b8ff203940ee5339 and project ae1d64a1d7b54a819ffa1c913bdcecaa. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 617.479847] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] [instance: 6f8e35f3-4b35-449c-9e60-1e0624f41cd2] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 617.480305] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 617.480678] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 617.480892] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 617.486616] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] [instance: 6f8e35f3-4b35-449c-9e60-1e0624f41cd2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 617.487478] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 617.487848] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 617.488081] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 617.509857] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 617.510095] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 617.510289] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 618.271747] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Expecting reply to msg f50578d62365472d9f6fb9c88fd5bebd in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 618.575829] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f50578d62365472d9f6fb9c88fd5bebd [ 618.774879] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Took 0.51 seconds to select destinations for 3 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 618.790544] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 618.791032] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 618.791263] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 618.836681] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 618.836681] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 618.836681] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 618.865304] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 618.865567] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 618.867025] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 618.916862] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 618.917153] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 618.917343] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 618.917730] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 618.917943] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 618.918114] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 618.927742] nova-conductor[54075]: DEBUG nova.quota [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Getting quotas for project 118f460d1ed848db9ce4526a46c64fdd. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 618.930894] nova-conductor[54075]: DEBUG nova.quota [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Getting quotas for user 82cc44e823e34c32bc9ad3c68f51c043 and project 118f460d1ed848db9ce4526a46c64fdd. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 618.942216] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: c9a6d28b-52f8-4636-886a-c74f0900e761] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 618.942723] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 618.942932] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 618.943102] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 618.950197] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: c9a6d28b-52f8-4636-886a-c74f0900e761] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 618.950864] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 618.951310] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 618.951487] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 618.989177] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 618.989446] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 618.989693] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 619.001830] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: 27b21d98-bf4d-47ee-84e8-d5a3d10d87a4] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 619.002315] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 619.002525] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 619.002696] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 619.010428] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: 27b21d98-bf4d-47ee-84e8-d5a3d10d87a4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 619.011156] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 619.011366] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 619.011535] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 619.029676] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 619.029915] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 619.030090] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 619.036741] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: 9f77d364-928f-4595-9253-8bb216b9215b] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 619.037239] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 619.037490] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 619.037752] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 619.042897] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: 9f77d364-928f-4595-9253-8bb216b9215b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 619.043579] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 619.043772] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 619.043936] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 619.066521] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 619.066827] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 619.067052] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 623.820126] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] [instance: d91c5dae-4ece-4718-a16b-534729f7ba49] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port f8b581d8-499a-4fb9-841d-9a1f886b5fda, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance d91c5dae-4ece-4718-a16b-534729f7ba49 was re-scheduled: Binding failed for port f8b581d8-499a-4fb9-841d-9a1f886b5fda, please check neutron logs for more information.\n'] [ 623.820747] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 623.820986] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d91c5dae-4ece-4718-a16b-534729f7ba49.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d91c5dae-4ece-4718-a16b-534729f7ba49. [ 623.821298] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] [instance: d91c5dae-4ece-4718-a16b-534729f7ba49] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d91c5dae-4ece-4718-a16b-534729f7ba49. [ 623.854849] nova-conductor[54074]: DEBUG nova.network.neutron [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] [instance: d91c5dae-4ece-4718-a16b-534729f7ba49] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 623.893252] nova-conductor[54074]: DEBUG nova.network.neutron [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] [instance: d91c5dae-4ece-4718-a16b-534729f7ba49] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 623.897603] nova-conductor[54074]: DEBUG nova.network.neutron [None req-54275bfb-afdc-49f7-a6da-f87693fe4de7 tempest-DeleteServersAdminTestJSON-911911890 tempest-DeleteServersAdminTestJSON-911911890-project-member] [instance: d91c5dae-4ece-4718-a16b-534729f7ba49] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 624.601539] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=8 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 624.639758] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=24,nova_cell1:SAVEPOINT=6,nova_cell1:INSERT=62,nova_cell1:RELEASE=6 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 624.649907] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=55,nova_api:UPDATE=4,nova_api:DELETE=3 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 624.651744] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=42,nova_api:UPDATE=4,nova_api:DELETE=5 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 627.002135] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Expecting reply to msg 3bb268dc248a491883e119fd1bd2178f in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 627.176103] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3bb268dc248a491883e119fd1bd2178f [ 627.506711] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 627.521340] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 627.521733] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 627.522007] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 627.572869] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 627.573319] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 627.580129] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 627.581702] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 627.583006] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 627.583334] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 627.603664] nova-conductor[54074]: DEBUG nova.quota [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Getting quotas for project 71a669cc62964b05bdaa71442c08a566. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 627.606682] nova-conductor[54074]: DEBUG nova.quota [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Getting quotas for user 2d003a5943d141cc9165e43148dec381 and project 71a669cc62964b05bdaa71442c08a566. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 627.614397] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 832c9ce1-6344-485a-a9ef-6950d1c78ef9] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 627.614851] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 627.615102] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 627.615283] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 627.623502] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 832c9ce1-6344-485a-a9ef-6950d1c78ef9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 627.624271] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 627.624490] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 627.624656] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 627.672118] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 627.672496] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 627.672496] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 628.484413] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Expecting reply to msg 7f4ed1cc94a64ade93ae9479b478828b in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 628.581668] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] [instance: 2fb7c1e4-d756-4528-914e-b924c5a3be38] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 398e9cea-37a5-4265-b56d-a19344b26784, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 2fb7c1e4-d756-4528-914e-b924c5a3be38 was re-scheduled: Binding failed for port 398e9cea-37a5-4265-b56d-a19344b26784, please check neutron logs for more information.\n'] [ 628.581668] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 628.581668] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2fb7c1e4-d756-4528-914e-b924c5a3be38.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2fb7c1e4-d756-4528-914e-b924c5a3be38. [ 628.581668] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] [instance: 2fb7c1e4-d756-4528-914e-b924c5a3be38] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2fb7c1e4-d756-4528-914e-b924c5a3be38. [ 628.614508] nova-conductor[54074]: DEBUG nova.network.neutron [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] [instance: 2fb7c1e4-d756-4528-914e-b924c5a3be38] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 628.650891] nova-conductor[54074]: DEBUG nova.network.neutron [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] [instance: 2fb7c1e4-d756-4528-914e-b924c5a3be38] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 628.654477] nova-conductor[54074]: DEBUG nova.network.neutron [None req-a2b2b44c-46a1-45bb-bf84-c88930f28c90 tempest-ImagesNegativeTestJSON-1745274774 tempest-ImagesNegativeTestJSON-1745274774-project-member] [instance: 2fb7c1e4-d756-4528-914e-b924c5a3be38] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 628.680540] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7f4ed1cc94a64ade93ae9479b478828b [ 628.923274] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=17 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 628.987047] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 629.040575] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 629.040737] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 629.040904] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 629.113617] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 629.113886] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 629.114027] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 629.114398] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 629.114580] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 629.114740] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 629.131920] nova-conductor[54075]: DEBUG nova.quota [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Getting quotas for project 9450f96e008846c39430838ba9f1ec08. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 629.136445] nova-conductor[54075]: DEBUG nova.quota [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Getting quotas for user bb5d486660a54e71a827e082e050bde5 and project 9450f96e008846c39430838ba9f1ec08. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 629.144798] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] [instance: 3d99c7df-b031-4187-988c-f642f79073d3] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 629.145327] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 629.145527] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 629.145687] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 629.150686] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] [instance: 3d99c7df-b031-4187-988c-f642f79073d3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 629.151379] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 629.151575] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 629.151736] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 629.222841] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 629.223151] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 629.223435] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 637.502611] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Expecting reply to msg 1ddbc03ff7de47edaeada654845f5494 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 637.592712] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 637.626891] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=3,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=8,nova_cell1:RELEASE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 637.645564] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=5,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 637.665496] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1ddbc03ff7de47edaeada654845f5494 [ 637.839027] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Expecting reply to msg 21a47db7d82240149fc94daf646f31f0 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 637.924170] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] [instance: f1239cdd-d1b3-4494-8204-0fe150737579] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 4f949fd6-9396-4f81-896d-140fe0cabbdf, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance f1239cdd-d1b3-4494-8204-0fe150737579 was re-scheduled: Binding failed for port 4f949fd6-9396-4f81-896d-140fe0cabbdf, please check neutron logs for more information.\n'] [ 637.924170] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 637.924170] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f1239cdd-d1b3-4494-8204-0fe150737579.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f1239cdd-d1b3-4494-8204-0fe150737579. [ 637.924170] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] [instance: f1239cdd-d1b3-4494-8204-0fe150737579] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f1239cdd-d1b3-4494-8204-0fe150737579. [ 637.969254] nova-conductor[54075]: DEBUG nova.network.neutron [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] [instance: f1239cdd-d1b3-4494-8204-0fe150737579] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 637.994166] nova-conductor[54075]: DEBUG nova.network.neutron [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] [instance: f1239cdd-d1b3-4494-8204-0fe150737579] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 638.002594] nova-conductor[54075]: DEBUG nova.network.neutron [None req-433c9b89-9ec3-4e6f-aa83-1de888b868a0 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] [instance: f1239cdd-d1b3-4494-8204-0fe150737579] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 638.007508] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 638.020234] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 21a47db7d82240149fc94daf646f31f0 [ 638.029553] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 638.029782] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 638.029951] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 638.077060] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 638.077060] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 638.077060] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 638.077209] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 638.077306] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 638.077454] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 638.094623] nova-conductor[54074]: DEBUG nova.quota [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Getting quotas for project 3d1ce7730c9d48cfb8a68754aad558ea. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 638.098311] nova-conductor[54074]: DEBUG nova.quota [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Getting quotas for user 86c0cc330fe84a8abab59cfe4ec905b1 and project 3d1ce7730c9d48cfb8a68754aad558ea. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 638.105413] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] [instance: 87304cf6-e65f-41de-ab6f-d2170aaa9064] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 638.106165] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 638.106511] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 638.106912] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 638.113099] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] [instance: 87304cf6-e65f-41de-ab6f-d2170aaa9064] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 638.114052] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 638.114379] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 638.114575] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 638.151402] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 638.151772] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 638.152081] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 638.341953] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 638.354397] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 638.354648] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 638.355282] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 638.391288] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 638.391635] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 638.391864] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 638.392299] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 638.392897] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 638.393108] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 638.407492] nova-conductor[54074]: DEBUG nova.quota [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Getting quotas for project 3d1ce7730c9d48cfb8a68754aad558ea. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 638.411175] nova-conductor[54074]: DEBUG nova.quota [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Getting quotas for user 86c0cc330fe84a8abab59cfe4ec905b1 and project 3d1ce7730c9d48cfb8a68754aad558ea. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 638.416014] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] [instance: 90c50f92-c1ff-4ac9-a819-ae0083884e28] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 638.416672] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 638.416883] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 638.417051] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 638.421451] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] [instance: 90c50f92-c1ff-4ac9-a819-ae0083884e28] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 638.422896] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 638.423353] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 638.423632] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 638.448898] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 638.449557] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 638.449866] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 639.118665] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 639.154159] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=58,nova_cell1:SAVEPOINT=16,nova_cell1:INSERT=157,nova_cell1:RELEASE=16 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 639.199135] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=113,nova_api:DELETE=11,nova_api:UPDATE=8 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 639.211583] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=111,nova_api:UPDATE=12,nova_api:DELETE=9 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 639.944437] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Expecting reply to msg 03e7605d4f734f4cb9905254be962db0 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 640.093917] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 03e7605d4f734f4cb9905254be962db0 [ 640.192351] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Expecting reply to msg 82a3f98029794eff822d206b03010ddb in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 640.206485] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] [instance: ce54ba3d-2cd5-4400-b334-8443ef73bbff] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port a786577c-7bcb-4451-83d7-073ce6e077ed, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ce54ba3d-2cd5-4400-b334-8443ef73bbff was re-scheduled: Binding failed for port a786577c-7bcb-4451-83d7-073ce6e077ed, please check neutron logs for more information.\n'] [ 640.206991] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 640.206991] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ce54ba3d-2cd5-4400-b334-8443ef73bbff.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ce54ba3d-2cd5-4400-b334-8443ef73bbff. [ 640.206991] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] [instance: ce54ba3d-2cd5-4400-b334-8443ef73bbff] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ce54ba3d-2cd5-4400-b334-8443ef73bbff. [ 640.241215] nova-conductor[54075]: DEBUG nova.network.neutron [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] [instance: ce54ba3d-2cd5-4400-b334-8443ef73bbff] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 640.271175] nova-conductor[54075]: DEBUG nova.network.neutron [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] [instance: ce54ba3d-2cd5-4400-b334-8443ef73bbff] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 640.278196] nova-conductor[54075]: DEBUG nova.network.neutron [None req-a4e1eb09-4885-4789-90e5-9152c14489d2 tempest-ServerPasswordTestJSON-1991055656 tempest-ServerPasswordTestJSON-1991055656-project-member] [instance: ce54ba3d-2cd5-4400-b334-8443ef73bbff] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 640.347356] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 82a3f98029794eff822d206b03010ddb [ 640.449092] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 640.467108] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.467333] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.467499] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 640.513957] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.514251] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.514641] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 640.515236] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.515668] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.516332] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 640.527805] nova-conductor[54075]: DEBUG nova.quota [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Getting quotas for project 4a940e96edf044a59d9882d228b2bb21. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 640.531306] nova-conductor[54075]: DEBUG nova.quota [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Getting quotas for user 394e29ffb261493197f3ef862213d774 and project 4a940e96edf044a59d9882d228b2bb21. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 640.536633] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] [instance: ae026dca-dc05-4710-8a03-4e792a0dc61d] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 640.537077] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.537282] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.537449] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 640.542650] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] [instance: ae026dca-dc05-4710-8a03-4e792a0dc61d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 640.543331] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.543546] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.543717] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 640.564534] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.564755] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.564920] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 640.694884] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 640.708135] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.708135] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.708135] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 640.731863] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.732306] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.732568] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 640.733065] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.733373] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.733858] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 640.742522] nova-conductor[54074]: DEBUG nova.quota [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Getting quotas for project d5eead97e11a4688b0058d50d5bef6ef. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 640.744961] nova-conductor[54074]: DEBUG nova.quota [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Getting quotas for user 332259bb7d2841e385dc46a558da2fd9 and project d5eead97e11a4688b0058d50d5bef6ef. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 640.750467] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] [instance: 16b04a1b-0ab3-4386-a1eb-74ef3e46a553] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 640.751187] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.751474] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.751747] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 640.764093] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] [instance: 16b04a1b-0ab3-4386-a1eb-74ef3e46a553] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 640.764994] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.765402] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.765786] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 640.779891] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.780282] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.780562] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.440518] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=382,nova_cell1:UPDATE=189,nova_cell1:SAVEPOINT=44,nova_cell1:RELEASE=44,nova_cell1:INSERT=31,nova_cell1:DELETE=4 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 644.800720] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] [instance: 5f4a5c62-85f1-47ee-b702-1785bfe62f48] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port cbcafabe-4af3-498d-b675-7d4cb156228f, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 5f4a5c62-85f1-47ee-b702-1785bfe62f48 was re-scheduled: Binding failed for port cbcafabe-4af3-498d-b675-7d4cb156228f, please check neutron logs for more information.\n'] [ 644.801301] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 644.801530] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5f4a5c62-85f1-47ee-b702-1785bfe62f48.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5f4a5c62-85f1-47ee-b702-1785bfe62f48. [ 644.801857] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] [instance: 5f4a5c62-85f1-47ee-b702-1785bfe62f48] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5f4a5c62-85f1-47ee-b702-1785bfe62f48. [ 644.830911] nova-conductor[54075]: DEBUG nova.network.neutron [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] [instance: 5f4a5c62-85f1-47ee-b702-1785bfe62f48] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 644.854672] nova-conductor[54075]: DEBUG nova.network.neutron [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] [instance: 5f4a5c62-85f1-47ee-b702-1785bfe62f48] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 644.857683] nova-conductor[54075]: DEBUG nova.network.neutron [None req-ae7f93d7-5a72-435a-8e7c-94046f28c0bf tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] [instance: 5f4a5c62-85f1-47ee-b702-1785bfe62f48] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 645.281179] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Expecting reply to msg 7e8914d22cda40d8b8643be92c5e9451 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 645.434164] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7e8914d22cda40d8b8643be92c5e9451 [ 645.784172] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 645.804115] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 645.804359] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 645.804489] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 645.855894] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 645.856718] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 645.856972] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 645.857370] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 645.857759] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 645.858101] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 645.881183] nova-conductor[54075]: DEBUG nova.quota [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Getting quotas for project 8d361e27776e47eeadaa4a29b4f9338f. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 645.886769] nova-conductor[54075]: DEBUG nova.quota [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Getting quotas for user 0933891c54584b059f68770a8c930f1d and project 8d361e27776e47eeadaa4a29b4f9338f. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 645.896757] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] [instance: 08638aac-2c6c-4580-9894-6b3b3c1ec484] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 645.897224] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 645.897441] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 645.897643] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 645.904660] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] [instance: 08638aac-2c6c-4580-9894-6b3b3c1ec484] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 645.905293] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 645.905495] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 645.905654] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 645.926625] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 645.926877] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 645.927733] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 646.878269] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] [instance: 56f9bb28-2770-46aa-9d95-f60cdeae0967] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 1d761917-ebce-43d1-b26b-0773dc004f35, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 56f9bb28-2770-46aa-9d95-f60cdeae0967 was re-scheduled: Binding failed for port 1d761917-ebce-43d1-b26b-0773dc004f35, please check neutron logs for more information.\n'] [ 646.879422] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 646.879660] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 56f9bb28-2770-46aa-9d95-f60cdeae0967.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 56f9bb28-2770-46aa-9d95-f60cdeae0967. [ 646.879972] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] [instance: 56f9bb28-2770-46aa-9d95-f60cdeae0967] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 56f9bb28-2770-46aa-9d95-f60cdeae0967. [ 646.901750] nova-conductor[54074]: DEBUG nova.network.neutron [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] [instance: 56f9bb28-2770-46aa-9d95-f60cdeae0967] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 646.930046] nova-conductor[54074]: DEBUG nova.network.neutron [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] [instance: 56f9bb28-2770-46aa-9d95-f60cdeae0967] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 646.934350] nova-conductor[54074]: DEBUG nova.network.neutron [None req-26b49dcd-b372-4629-ab78-5c002c219e38 tempest-VolumesAdminNegativeTest-459345843 tempest-VolumesAdminNegativeTest-459345843-project-member] [instance: 56f9bb28-2770-46aa-9d95-f60cdeae0967] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 647.195780] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=405,nova_cell1:SAVEPOINT=39,nova_cell1:RELEASE=39,nova_cell1:UPDATE=189,nova_cell1:INSERT=21 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 650.062336] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Expecting reply to msg a2876f32c61f4103be68fe82930e14b2 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 650.212730] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a2876f32c61f4103be68fe82930e14b2 [ 650.568774] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 650.584850] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 650.584850] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 650.584850] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 650.631381] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 650.631697] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 650.631911] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 650.632358] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 650.632583] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 650.632782] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 650.643332] nova-conductor[54074]: DEBUG nova.quota [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Getting quotas for project 003b76634a314c8ba271d8ff8a84f4cb. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 650.647021] nova-conductor[54074]: DEBUG nova.quota [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Getting quotas for user 977af06e03fb4180b6c5cfd5ae8aa0cf and project 003b76634a314c8ba271d8ff8a84f4cb. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 650.661814] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] [instance: 66a0a424-ecb6-43df-9b47-946ff1e1b7b2] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 650.662326] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 650.664281] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 650.664281] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 650.677745] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] [instance: 66a0a424-ecb6-43df-9b47-946ff1e1b7b2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 650.678440] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 650.678648] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 650.678815] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 650.695819] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 650.696142] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 650.696248] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 653.554080] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] [instance: 35411b03-ace3-40da-8c3e-3872ac003bd3] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 4757bd66-fea5-408a-b5a7-7b719b899f50, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 35411b03-ace3-40da-8c3e-3872ac003bd3 was re-scheduled: Binding failed for port 4757bd66-fea5-408a-b5a7-7b719b899f50, please check neutron logs for more information.\n'] [ 653.554654] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 653.554907] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 35411b03-ace3-40da-8c3e-3872ac003bd3.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 35411b03-ace3-40da-8c3e-3872ac003bd3. [ 653.555808] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] [instance: 35411b03-ace3-40da-8c3e-3872ac003bd3] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 35411b03-ace3-40da-8c3e-3872ac003bd3. [ 653.579290] nova-conductor[54075]: DEBUG nova.network.neutron [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] [instance: 35411b03-ace3-40da-8c3e-3872ac003bd3] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 653.593802] nova-conductor[54075]: DEBUG nova.network.neutron [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] [instance: 35411b03-ace3-40da-8c3e-3872ac003bd3] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 653.598913] nova-conductor[54075]: DEBUG nova.network.neutron [None req-28341f39-563b-434f-9742-0ac8f24e40ab tempest-ServersTestManualDisk-2065425642 tempest-ServersTestManualDisk-2065425642-project-member] [instance: 35411b03-ace3-40da-8c3e-3872ac003bd3] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 655.664062] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] [instance: 3454f8a4-a9c7-4622-9cdc-36f683b6f3ee] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 0a638f3f-d212-4e30-8d09-d277b8a79f1f, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 3454f8a4-a9c7-4622-9cdc-36f683b6f3ee was re-scheduled: Binding failed for port 0a638f3f-d212-4e30-8d09-d277b8a79f1f, please check neutron logs for more information.\n'] [ 655.664062] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 655.664062] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3454f8a4-a9c7-4622-9cdc-36f683b6f3ee.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3454f8a4-a9c7-4622-9cdc-36f683b6f3ee. [ 655.664062] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] [instance: 3454f8a4-a9c7-4622-9cdc-36f683b6f3ee] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3454f8a4-a9c7-4622-9cdc-36f683b6f3ee. [ 655.690680] nova-conductor[54075]: DEBUG nova.network.neutron [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] [instance: 3454f8a4-a9c7-4622-9cdc-36f683b6f3ee] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 655.704942] nova-conductor[54075]: DEBUG nova.network.neutron [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] [instance: 3454f8a4-a9c7-4622-9cdc-36f683b6f3ee] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 655.708405] nova-conductor[54075]: DEBUG nova.network.neutron [None req-d2fd4c59-8101-4bad-b7a3-afdc2994a4b5 tempest-ImagesOneServerTestJSON-1598717001 tempest-ImagesOneServerTestJSON-1598717001-project-member] [instance: 3454f8a4-a9c7-4622-9cdc-36f683b6f3ee] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 655.865983] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=2 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 655.908404] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=6,nova_cell1:SAVEPOINT=2,nova_cell1:INSERT=16,nova_cell1:RELEASE=2 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 655.921442] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=12,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 655.922784] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=12,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 656.818459] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Expecting reply to msg 60ce988c46af437f99c2b1cc16aa06fd in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 656.954236] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 60ce988c46af437f99c2b1cc16aa06fd [ 657.321527] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 657.336262] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 657.336482] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 657.336766] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 657.362096] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Expecting reply to msg fc37cad83c674502b973f40e52a958b9 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 657.370773] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 657.371138] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 657.371407] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 657.371928] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 657.372392] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 657.372659] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 657.383672] nova-conductor[54074]: DEBUG nova.quota [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Getting quotas for project f9cb6ab6d9d64c6eb3d4d8d37563f6cc. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 657.386039] nova-conductor[54074]: DEBUG nova.quota [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Getting quotas for user b0c3dc5a397c4f1e9118b36d504166f5 and project f9cb6ab6d9d64c6eb3d4d8d37563f6cc. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 657.391021] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: e8c77459-e3a3-4a68-9f76-0757dd0f2587] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 657.391674] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 657.391983] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 657.392277] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 657.397785] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: e8c77459-e3a3-4a68-9f76-0757dd0f2587] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 657.398765] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 657.399203] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 657.399930] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 657.416589] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 657.416936] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 657.417200] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 657.499015] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fc37cad83c674502b973f40e52a958b9 [ 657.692067] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Expecting reply to msg ede49758faf24ee79d01a22859f9ac96 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 657.819845] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ede49758faf24ee79d01a22859f9ac96 [ 657.865204] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 657.880943] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 657.881259] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 657.881792] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 657.916207] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 657.916445] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 657.916617] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 657.916987] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 657.917166] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 657.917320] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 657.931736] nova-conductor[54075]: DEBUG nova.quota [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Getting quotas for project f9cb6ab6d9d64c6eb3d4d8d37563f6cc. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 657.934535] nova-conductor[54075]: DEBUG nova.quota [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Getting quotas for user b0c3dc5a397c4f1e9118b36d504166f5 and project f9cb6ab6d9d64c6eb3d4d8d37563f6cc. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 657.939453] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: 9a1c4327-64b3-4c4d-b6ae-77959084b405] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 657.939908] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 657.940135] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 657.940346] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 657.945300] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: 9a1c4327-64b3-4c4d-b6ae-77959084b405] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 657.945963] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 657.946161] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 657.946324] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 657.974523] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 657.974792] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 657.974932] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 658.194672] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 658.214311] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 658.214738] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 658.215013] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 658.251757] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 658.251966] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 658.252153] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 658.252523] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 658.252703] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 658.252860] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 658.268796] nova-conductor[54075]: DEBUG nova.quota [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Getting quotas for project f9cb6ab6d9d64c6eb3d4d8d37563f6cc. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 658.271826] nova-conductor[54075]: DEBUG nova.quota [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Getting quotas for user b0c3dc5a397c4f1e9118b36d504166f5 and project f9cb6ab6d9d64c6eb3d4d8d37563f6cc. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 658.279211] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: a6ec5486-0843-4c38-b187-35d5296965a7] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 658.279706] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 658.279916] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 658.280098] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 658.287916] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: a6ec5486-0843-4c38-b187-35d5296965a7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 658.290045] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 658.290296] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 658.290494] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 658.306333] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 658.306564] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 658.306739] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 662.578515] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Expecting reply to msg 9c962e5768874554ab60e77b688bff42 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 662.721153] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9c962e5768874554ab60e77b688bff42 [ 663.081047] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 663.102786] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 663.103359] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 663.103679] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 663.154142] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 663.154386] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 663.154629] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 663.155444] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 663.155444] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 663.155699] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 663.170522] nova-conductor[54075]: DEBUG nova.quota [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Getting quotas for project 0a8aeaac48ad42878d34975953c280fc. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 663.173881] nova-conductor[54075]: DEBUG nova.quota [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Getting quotas for user a6d97155b2a64df88fa3c3885c771c64 and project 0a8aeaac48ad42878d34975953c280fc. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 663.181401] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] [instance: c7ec619c-1b00-4d58-a593-671c0139c4e3] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 663.181922] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 663.182145] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 663.182326] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 663.191819] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] [instance: c7ec619c-1b00-4d58-a593-671c0139c4e3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 663.192924] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 663.193186] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 663.193360] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 663.212354] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 663.212624] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 663.212804] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 667.378136] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=5 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 667.402998] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=15,nova_cell1:SAVEPOINT=2,nova_cell1:INSERT=37,nova_cell1:RELEASE=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 667.410826] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=24,nova_api:DELETE=4,nova_api:UPDATE=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 667.414158] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=43,nova_api:UPDATE=3,nova_api:DELETE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 673.159599] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=3 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 673.195289] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=23,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 673.205980] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=18,nova_api:UPDATE=2 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 673.208608] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=18,nova_api:DELETE=3,nova_api:UPDATE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 675.719771] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] [instance: fa25ef8c-cb4a-4c5b-8157-3d2e1a2e3932] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port a8f53cf7-372e-482c-94e0-6ed3a6e4442a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance fa25ef8c-cb4a-4c5b-8157-3d2e1a2e3932 was re-scheduled: Binding failed for port a8f53cf7-372e-482c-94e0-6ed3a6e4442a, please check neutron logs for more information.\n'] [ 675.720357] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 675.720589] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fa25ef8c-cb4a-4c5b-8157-3d2e1a2e3932.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fa25ef8c-cb4a-4c5b-8157-3d2e1a2e3932. [ 675.720926] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] [instance: fa25ef8c-cb4a-4c5b-8157-3d2e1a2e3932] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fa25ef8c-cb4a-4c5b-8157-3d2e1a2e3932. [ 675.741396] nova-conductor[54074]: DEBUG nova.network.neutron [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] [instance: fa25ef8c-cb4a-4c5b-8157-3d2e1a2e3932] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 675.756229] nova-conductor[54074]: DEBUG nova.network.neutron [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] [instance: fa25ef8c-cb4a-4c5b-8157-3d2e1a2e3932] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 675.759316] nova-conductor[54074]: DEBUG nova.network.neutron [None req-63a73b02-9dca-4d4a-9b22-3f51d1c4763c tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] [instance: fa25ef8c-cb4a-4c5b-8157-3d2e1a2e3932] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 678.713807] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Expecting reply to msg f407195b96b64449a162e0c45b03f28c in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 678.853922] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f407195b96b64449a162e0c45b03f28c [ 679.217286] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 679.230437] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 679.230714] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 679.230925] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 679.295052] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 679.295270] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 679.295496] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 679.295987] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 679.296200] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 679.296464] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 679.306825] nova-conductor[54074]: DEBUG nova.quota [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Getting quotas for project f74de90347d44395b052d75738d6a065. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 679.309663] nova-conductor[54074]: DEBUG nova.quota [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Getting quotas for user 2857ea8202fc46b09da06c4ac904df95 and project f74de90347d44395b052d75738d6a065. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 679.314344] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] [instance: a04d014e-bed6-4e4b-a5eb-316d88c174f0] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 679.314750] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 679.314946] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 679.315114] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 679.319539] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] [instance: a04d014e-bed6-4e4b-a5eb-316d88c174f0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 679.320160] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 679.320379] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 679.320546] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 679.340649] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 679.340857] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 679.341021] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 680.291233] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] [instance: 80a15f60-4843-4a59-a6c1-0d5624609672] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 137bb12e-27d8-4868-8174-3f7459cea1f9, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 80a15f60-4843-4a59-a6c1-0d5624609672 was re-scheduled: Binding failed for port 137bb12e-27d8-4868-8174-3f7459cea1f9, please check neutron logs for more information.\n'] [ 680.292328] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 680.292576] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 80a15f60-4843-4a59-a6c1-0d5624609672.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 80a15f60-4843-4a59-a6c1-0d5624609672. [ 680.293375] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] [instance: 80a15f60-4843-4a59-a6c1-0d5624609672] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 80a15f60-4843-4a59-a6c1-0d5624609672. [ 680.319177] nova-conductor[54074]: DEBUG nova.network.neutron [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] [instance: 80a15f60-4843-4a59-a6c1-0d5624609672] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 680.352613] nova-conductor[54074]: DEBUG nova.network.neutron [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] [instance: 80a15f60-4843-4a59-a6c1-0d5624609672] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 680.356176] nova-conductor[54074]: DEBUG nova.network.neutron [None req-313e25df-76e7-442e-b13f-292f73b86b63 tempest-InstanceActionsTestJSON-1004074533 tempest-InstanceActionsTestJSON-1004074533-project-member] [instance: 80a15f60-4843-4a59-a6c1-0d5624609672] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 684.747872] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Expecting reply to msg 993a52c53cd8454ba2b4ddd6d4dca305 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 684.907153] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 993a52c53cd8454ba2b4ddd6d4dca305 [ 685.001732] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Expecting reply to msg abd3e9fe09d34b338892d0ece8362fae in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 685.155409] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg abd3e9fe09d34b338892d0ece8362fae [ 685.250797] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 685.266012] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 685.266259] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 685.266431] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 685.298600] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 685.298825] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 685.299037] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 685.299409] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 685.299612] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 685.299750] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 685.309017] nova-conductor[54075]: DEBUG nova.quota [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Getting quotas for project c0b7ebae5346428b85094247a63eb203. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 685.312167] nova-conductor[54075]: DEBUG nova.quota [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Getting quotas for user d989fd9db5e94df89beee55f4aa08bf5 and project c0b7ebae5346428b85094247a63eb203. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 685.317382] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] [instance: b95c60dc-50c4-4afc-acb0-3308e490b808] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 685.317927] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 685.318136] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 685.318313] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 685.322895] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] [instance: b95c60dc-50c4-4afc-acb0-3308e490b808] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 685.323504] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 685.323709] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 685.323877] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 685.342972] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 685.343199] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 685.343368] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-a8cd4b97-b5aa-417a-8297-971fb71887bb tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 685.506736] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 685.519169] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 685.519404] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 685.519573] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 685.565182] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 685.565531] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 685.565814] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 685.566335] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 685.566635] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 685.566909] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 685.575403] nova-conductor[54074]: DEBUG nova.quota [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Getting quotas for project c0b7ebae5346428b85094247a63eb203. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 685.577849] nova-conductor[54074]: DEBUG nova.quota [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Getting quotas for user d989fd9db5e94df89beee55f4aa08bf5 and project c0b7ebae5346428b85094247a63eb203. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 685.583206] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] [instance: 900e1e1e-5635-4782-bd87-046dd2af7dad] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 685.583767] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 685.584110] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 685.584389] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 685.587119] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] [instance: 900e1e1e-5635-4782-bd87-046dd2af7dad] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 685.588361] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 685.589213] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 685.589638] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 685.604384] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 685.604603] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 685.604768] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-894882f9-f909-4f49-a03f-0e4f67e2b6c5 tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 687.241726] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] [instance: 15e2e743-070f-4545-b976-ced38fd99198] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 530e4dd8-07d2-480d-9d84-700373a8d44a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 15e2e743-070f-4545-b976-ced38fd99198 was re-scheduled: Binding failed for port 530e4dd8-07d2-480d-9d84-700373a8d44a, please check neutron logs for more information.\n'] [ 687.242444] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 687.242697] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 15e2e743-070f-4545-b976-ced38fd99198.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 15e2e743-070f-4545-b976-ced38fd99198. [ 687.243000] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] [instance: 15e2e743-070f-4545-b976-ced38fd99198] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 15e2e743-070f-4545-b976-ced38fd99198. [ 687.267129] nova-conductor[54075]: DEBUG nova.network.neutron [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] [instance: 15e2e743-070f-4545-b976-ced38fd99198] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 687.284575] nova-conductor[54075]: DEBUG nova.network.neutron [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] [instance: 15e2e743-070f-4545-b976-ced38fd99198] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 687.289549] nova-conductor[54075]: DEBUG nova.network.neutron [None req-39d251b7-0311-4e19-ba29-53bcbff7e5f0 tempest-ServerExternalEventsTest-1156907895 tempest-ServerExternalEventsTest-1156907895-project-member] [instance: 15e2e743-070f-4545-b976-ced38fd99198] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 689.332155] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] [instance: 2fa640c2-b433-4581-be4b-0673c1451043] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 59e1287a-ba70-419b-a589-4ff8cd85c074, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 2fa640c2-b433-4581-be4b-0673c1451043 was re-scheduled: Binding failed for port 59e1287a-ba70-419b-a589-4ff8cd85c074, please check neutron logs for more information.\n'] [ 689.332819] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 689.333054] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2fa640c2-b433-4581-be4b-0673c1451043.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2fa640c2-b433-4581-be4b-0673c1451043. [ 689.333319] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] [instance: 2fa640c2-b433-4581-be4b-0673c1451043] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2fa640c2-b433-4581-be4b-0673c1451043. [ 689.357070] nova-conductor[54075]: DEBUG nova.network.neutron [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] [instance: 2fa640c2-b433-4581-be4b-0673c1451043] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 689.376584] nova-conductor[54075]: DEBUG nova.network.neutron [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] [instance: 2fa640c2-b433-4581-be4b-0673c1451043] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 689.380100] nova-conductor[54075]: DEBUG nova.network.neutron [None req-0db3a869-c29c-41b3-92ea-8e7694544b1e tempest-AttachInterfacesV270Test-560058959 tempest-AttachInterfacesV270Test-560058959-project-member] [instance: 2fa640c2-b433-4581-be4b-0673c1451043] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 693.569377] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Expecting reply to msg fad5b481484b41ea9ae4ecc6a673a358 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 693.737907] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fad5b481484b41ea9ae4ecc6a673a358 [ 694.072361] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 694.088894] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 694.088894] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 694.088894] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 694.142700] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 694.143370] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 694.143660] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 694.144115] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 694.144317] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 694.144482] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 694.156425] nova-conductor[54074]: DEBUG nova.quota [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Getting quotas for project 4392957210d14f89af11bf7b1bf7ffc7. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 694.160351] nova-conductor[54074]: DEBUG nova.quota [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Getting quotas for user 85bae9b15b0f43c0bc6e4d4f4c6a28e3 and project 4392957210d14f89af11bf7b1bf7ffc7. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 694.167433] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] [instance: afc5587e-7fd5-4b07-aff8-98ef8358985f] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 694.168375] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 694.168632] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 694.168809] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 694.176117] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] [instance: afc5587e-7fd5-4b07-aff8-98ef8358985f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 694.177170] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 694.177474] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 694.177745] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 694.199561] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 694.200283] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 694.200644] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 695.303726] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 695.326025] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=3,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=8,nova_cell1:RELEASE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 695.334799] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=7,nova_api:UPDATE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 695.337645] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=6,nova_api:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 696.494996] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 535045d9-108e-4e88-82f0-9da98f2f55a6] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port d1298831-ff8d-4c39-806b-3bb7c2472352, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 535045d9-108e-4e88-82f0-9da98f2f55a6 was re-scheduled: Binding failed for port d1298831-ff8d-4c39-806b-3bb7c2472352, please check neutron logs for more information.\n'] [ 696.495649] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 696.495890] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 535045d9-108e-4e88-82f0-9da98f2f55a6.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 535045d9-108e-4e88-82f0-9da98f2f55a6. [ 696.496225] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 535045d9-108e-4e88-82f0-9da98f2f55a6] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 535045d9-108e-4e88-82f0-9da98f2f55a6. [ 696.518844] nova-conductor[54074]: DEBUG nova.network.neutron [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 535045d9-108e-4e88-82f0-9da98f2f55a6] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 696.546736] nova-conductor[54074]: DEBUG nova.network.neutron [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 535045d9-108e-4e88-82f0-9da98f2f55a6] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 696.550725] nova-conductor[54074]: DEBUG nova.network.neutron [None req-6a435c00-b673-481f-90a8-c507ab0df46a tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 535045d9-108e-4e88-82f0-9da98f2f55a6] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 697.097553] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Expecting reply to msg eedae6fdf6a6434daca5157d496e031a in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 697.255327] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg eedae6fdf6a6434daca5157d496e031a [ 697.601216] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 697.619872] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 697.620131] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 697.620346] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 697.660136] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 697.660376] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 697.660538] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 697.660907] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 697.661084] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 697.661240] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 697.671496] nova-conductor[54075]: DEBUG nova.quota [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Getting quotas for project 648690c514264bdba16759f1dce64e4b. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 697.674730] nova-conductor[54075]: DEBUG nova.quota [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Getting quotas for user aecef81773a14855b22e49bfdee52c23 and project 648690c514264bdba16759f1dce64e4b. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 697.679648] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] [instance: 47b83dbe-d7d8-4875-bb79-95a8fecf4028] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 697.680093] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 697.680331] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 697.680513] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 697.685164] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] [instance: 47b83dbe-d7d8-4875-bb79-95a8fecf4028] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 697.685800] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 697.686109] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 697.686276] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 697.703595] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 697.703824] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 697.703992] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 698.398230] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Expecting reply to msg b1fc3f47da9941bb8d70338e2225c10d in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 698.544750] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b1fc3f47da9941bb8d70338e2225c10d [ 698.916531] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Took 0.52 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 698.929872] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 698.930162] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 698.930401] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 698.970948] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 698.971188] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 698.971357] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 698.971724] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 698.971907] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 698.972094] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 698.993626] nova-conductor[54075]: DEBUG nova.quota [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Getting quotas for project 61a866168186462d9d849072a1ff25f1. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 698.997948] nova-conductor[54075]: DEBUG nova.quota [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Getting quotas for user 6349e1aff7d945a6a471b1f4e826b23f and project 61a866168186462d9d849072a1ff25f1. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 699.004483] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: aa1afca5-8194-4a9d-bcd0-e3e91c15338c] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 699.005314] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 699.005596] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 699.005924] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 699.011566] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: aa1afca5-8194-4a9d-bcd0-e3e91c15338c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 699.012347] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 699.012664] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 699.012960] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 699.041677] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 699.041677] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 699.041677] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 701.186451] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] [instance: 83a25ff9-cc7d-4917-95cc-e621884bcee8] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 033d9bed-a864-43c3-8c4b-838811ea8db1, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 83a25ff9-cc7d-4917-95cc-e621884bcee8 was re-scheduled: Binding failed for port 033d9bed-a864-43c3-8c4b-838811ea8db1, please check neutron logs for more information.\n'] [ 701.187172] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 701.187505] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 83a25ff9-cc7d-4917-95cc-e621884bcee8.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 83a25ff9-cc7d-4917-95cc-e621884bcee8. [ 701.187860] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] [instance: 83a25ff9-cc7d-4917-95cc-e621884bcee8] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 83a25ff9-cc7d-4917-95cc-e621884bcee8. [ 701.216374] nova-conductor[54075]: DEBUG nova.network.neutron [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] [instance: 83a25ff9-cc7d-4917-95cc-e621884bcee8] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 701.265067] nova-conductor[54075]: DEBUG nova.network.neutron [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] [instance: 83a25ff9-cc7d-4917-95cc-e621884bcee8] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 701.268529] nova-conductor[54075]: DEBUG nova.network.neutron [None req-d5bc7237-6f46-40f5-8d1a-1d1435f25dcb tempest-ServerGroupTestJSON-850352494 tempest-ServerGroupTestJSON-850352494-project-member] [instance: 83a25ff9-cc7d-4917-95cc-e621884bcee8] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 703.484236] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SAVEPOINT=28,nova_cell1:RELEASE=28,nova_cell1:SELECT=274,nova_cell1:UPDATE=127,nova_cell1:INSERT=22 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 704.150778] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=3 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 704.180647] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=22,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 704.193074] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=19,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 704.195266] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=17,nova_api:UPDATE=2,nova_api:DELETE=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 705.718776] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] [instance: b537150e-9136-4fa4-b092-4f4995b918b7] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 9b5b0d54-31e6-49e7-a134-cfc1557f1a7c, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance b537150e-9136-4fa4-b092-4f4995b918b7 was re-scheduled: Binding failed for port 9b5b0d54-31e6-49e7-a134-cfc1557f1a7c, please check neutron logs for more information.\n'] [ 705.718776] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 705.718776] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b537150e-9136-4fa4-b092-4f4995b918b7.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b537150e-9136-4fa4-b092-4f4995b918b7. [ 705.718776] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] [instance: b537150e-9136-4fa4-b092-4f4995b918b7] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b537150e-9136-4fa4-b092-4f4995b918b7. [ 705.744464] nova-conductor[54074]: DEBUG nova.network.neutron [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] [instance: b537150e-9136-4fa4-b092-4f4995b918b7] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 705.764609] nova-conductor[54074]: DEBUG nova.network.neutron [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] [instance: b537150e-9136-4fa4-b092-4f4995b918b7] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 705.768808] nova-conductor[54074]: DEBUG nova.network.neutron [None req-a3640907-da62-4cf8-89d4-91cd771388ef tempest-ServerDiagnosticsNegativeTest-1465321625 tempest-ServerDiagnosticsNegativeTest-1465321625-project-member] [instance: b537150e-9136-4fa4-b092-4f4995b918b7] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 707.528084] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=310,nova_cell1:UPDATE=140,nova_cell1:INSERT=18,nova_cell1:SAVEPOINT=31,nova_cell1:RELEASE=31 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 707.622953] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Expecting reply to msg 8b31c12047bd4a0c96b39081ecb00bb5 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 707.775938] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8b31c12047bd4a0c96b39081ecb00bb5 [ 707.789066] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] [instance: fe9756ba-0eb5-41ad-913f-e933f97542cb] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 8b1944f9-bd34-42ed-a99d-9be9225ffaf1, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance fe9756ba-0eb5-41ad-913f-e933f97542cb was re-scheduled: Binding failed for port 8b1944f9-bd34-42ed-a99d-9be9225ffaf1, please check neutron logs for more information.\n'] [ 707.790057] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 707.790057] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fe9756ba-0eb5-41ad-913f-e933f97542cb.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fe9756ba-0eb5-41ad-913f-e933f97542cb. [ 707.790217] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] [instance: fe9756ba-0eb5-41ad-913f-e933f97542cb] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fe9756ba-0eb5-41ad-913f-e933f97542cb. [ 707.819746] nova-conductor[54075]: DEBUG nova.network.neutron [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] [instance: fe9756ba-0eb5-41ad-913f-e933f97542cb] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 707.837042] nova-conductor[54075]: DEBUG nova.network.neutron [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] [instance: fe9756ba-0eb5-41ad-913f-e933f97542cb] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 707.841627] nova-conductor[54075]: DEBUG nova.network.neutron [None req-8de69702-3c2b-4bac-bfeb-b8b231ed5eb6 tempest-ImagesOneServerNegativeTestJSON-1955627597 tempest-ImagesOneServerNegativeTestJSON-1955627597-project-member] [instance: fe9756ba-0eb5-41ad-913f-e933f97542cb] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 708.125397] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 708.147948] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 708.148287] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 708.148510] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 708.192099] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 708.192381] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 708.192639] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 708.192952] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 708.193142] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 708.193317] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 708.206254] nova-conductor[54074]: DEBUG nova.quota [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Getting quotas for project a1ff92bed39f4c93ad77ea3d3e398a4a. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 708.209437] nova-conductor[54074]: DEBUG nova.quota [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Getting quotas for user 725320be5ef84e56a02daff05e075ac5 and project a1ff92bed39f4c93ad77ea3d3e398a4a. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 708.215511] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] [instance: 732cf1e3-823d-4769-ad16-f5b492be53d5] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 708.216540] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 708.216540] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 708.216540] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 708.228887] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] [instance: 732cf1e3-823d-4769-ad16-f5b492be53d5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 708.229659] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 708.229866] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 708.230035] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 708.250678] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 708.250935] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 708.251107] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 708.976634] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=2 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 709.017188] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=6,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=15,nova_cell1:RELEASE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 709.022450] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=11,nova_api:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 709.035369] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 713.958239] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Expecting reply to msg b04e3d50b7d0468bbccd8bebce3ac09d in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 714.095435] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b04e3d50b7d0468bbccd8bebce3ac09d [ 714.461938] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 714.495338] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 714.495566] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 714.495730] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 714.549905] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 714.550127] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 714.550363] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 714.550750] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 714.550960] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 714.551468] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 714.562778] nova-conductor[54075]: DEBUG nova.quota [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Getting quotas for project ea0635c0a7d846df93cd8dd342486acd. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 714.565655] nova-conductor[54075]: DEBUG nova.quota [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Getting quotas for user 793e0ffc9f9c447da4cefaf315b15eb7 and project ea0635c0a7d846df93cd8dd342486acd. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 714.571150] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] [instance: 436788b9-92bb-4088-9c24-c2e9a073c09d] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 714.571626] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 714.571848] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 714.572046] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 714.576563] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] [instance: 436788b9-92bb-4088-9c24-c2e9a073c09d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 714.577354] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 714.577484] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 714.577644] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 714.599324] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 714.599626] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 714.599893] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 715.013784] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] [instance: 11a6eaa1-0d35-49cf-9341-b74129cf087b] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b1bc0a4e-9fde-485b-9637-ec9f81514cd0, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 11a6eaa1-0d35-49cf-9341-b74129cf087b was re-scheduled: Binding failed for port b1bc0a4e-9fde-485b-9637-ec9f81514cd0, please check neutron logs for more information.\n'] [ 715.014412] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 715.014641] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 11a6eaa1-0d35-49cf-9341-b74129cf087b.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 11a6eaa1-0d35-49cf-9341-b74129cf087b. [ 715.014948] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] [instance: 11a6eaa1-0d35-49cf-9341-b74129cf087b] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 11a6eaa1-0d35-49cf-9341-b74129cf087b. [ 715.041271] nova-conductor[54075]: DEBUG nova.network.neutron [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] [instance: 11a6eaa1-0d35-49cf-9341-b74129cf087b] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 715.059506] nova-conductor[54075]: DEBUG nova.network.neutron [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] [instance: 11a6eaa1-0d35-49cf-9341-b74129cf087b] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 715.062842] nova-conductor[54075]: DEBUG nova.network.neutron [None req-53233ff4-749b-4f63-8c48-18354418014f tempest-InstanceActionsNegativeTestJSON-308951544 tempest-InstanceActionsNegativeTestJSON-308951544-project-member] [instance: 11a6eaa1-0d35-49cf-9341-b74129cf087b] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 716.605544] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Expecting reply to msg 799b005aca5e4bb49c8571b61b18df34 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 716.753249] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 799b005aca5e4bb49c8571b61b18df34 [ 716.928883] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 3a4e1dcc-610f-4037-94e9-c9815c12ed1d] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 2ee61c01-65fa-4dc1-b14e-99dd8e88ccfe, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 3a4e1dcc-610f-4037-94e9-c9815c12ed1d was re-scheduled: Binding failed for port 2ee61c01-65fa-4dc1-b14e-99dd8e88ccfe, please check neutron logs for more information.\n'] [ 716.929550] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 716.929795] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3a4e1dcc-610f-4037-94e9-c9815c12ed1d.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3a4e1dcc-610f-4037-94e9-c9815c12ed1d. [ 716.930143] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 3a4e1dcc-610f-4037-94e9-c9815c12ed1d] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3a4e1dcc-610f-4037-94e9-c9815c12ed1d. [ 716.955971] nova-conductor[54074]: DEBUG nova.network.neutron [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 3a4e1dcc-610f-4037-94e9-c9815c12ed1d] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 716.983025] nova-conductor[54074]: DEBUG nova.network.neutron [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 3a4e1dcc-610f-4037-94e9-c9815c12ed1d] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 716.993718] nova-conductor[54074]: DEBUG nova.network.neutron [None req-cb601abf-819c-4685-a22f-8b0340cfd065 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 3a4e1dcc-610f-4037-94e9-c9815c12ed1d] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 717.111036] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 717.133199] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 717.133199] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 717.133375] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 717.198232] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 717.198232] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 717.198232] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 717.198232] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 717.198232] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 717.198232] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 717.216165] nova-conductor[54074]: DEBUG nova.quota [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Getting quotas for project aa39a5fecdee4b9ab51732c89aeb5004. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 717.221155] nova-conductor[54074]: DEBUG nova.quota [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Getting quotas for user 4c96ff95d54342919bf01d662a5cf413 and project aa39a5fecdee4b9ab51732c89aeb5004. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 717.230203] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] [instance: 53d6d89d-04bb-421d-994c-014830491dfa] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 717.230878] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 717.231253] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 717.231596] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 717.254299] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] [instance: 53d6d89d-04bb-421d-994c-014830491dfa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 717.256422] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 717.256814] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 717.257254] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 717.296048] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 717.296537] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 717.296537] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 719.888348] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Expecting reply to msg c42fa710b6c44e24afb1e61c33cac320 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 720.040678] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c42fa710b6c44e24afb1e61c33cac320 [ 720.390960] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 720.411274] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 720.411274] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 720.411274] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 720.451077] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 720.451366] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 720.451537] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 720.451911] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 720.452141] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 720.452319] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 720.479276] nova-conductor[54074]: DEBUG nova.quota [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Getting quotas for project 21cdf8675fb347c2874d912dcb8ac002. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 720.482319] nova-conductor[54074]: DEBUG nova.quota [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Getting quotas for user 23f9946393284244aca0c29201c37fa4 and project 21cdf8675fb347c2874d912dcb8ac002. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 720.488562] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 3ada5090-7219-4835-b508-2188501ae5e4] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 720.489208] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 720.489565] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 720.489872] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 720.495528] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 3ada5090-7219-4835-b508-2188501ae5e4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 720.496625] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 720.496987] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 720.497298] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 720.515872] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 720.516767] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 720.517117] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 721.693265] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 0e018d70-d6dd-4f79-bb03-14b815645562] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 0dbb80c9-2197-47a4-942f-1135d1c701d1, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 0e018d70-d6dd-4f79-bb03-14b815645562 was re-scheduled: Binding failed for port 0dbb80c9-2197-47a4-942f-1135d1c701d1, please check neutron logs for more information.\n'] [ 721.693887] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 721.694364] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0e018d70-d6dd-4f79-bb03-14b815645562.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0e018d70-d6dd-4f79-bb03-14b815645562. [ 721.694712] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 0e018d70-d6dd-4f79-bb03-14b815645562] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0e018d70-d6dd-4f79-bb03-14b815645562. [ 721.716849] nova-conductor[54074]: DEBUG nova.network.neutron [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 0e018d70-d6dd-4f79-bb03-14b815645562] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 721.738905] nova-conductor[54074]: DEBUG nova.network.neutron [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 0e018d70-d6dd-4f79-bb03-14b815645562] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 721.742639] nova-conductor[54074]: DEBUG nova.network.neutron [None req-f65890f8-d697-442e-af8a-c53a9fdd2611 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 0e018d70-d6dd-4f79-bb03-14b815645562] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 723.205385] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-ad36f3a0-38c5-4b94-834f-5999e98f33fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Expecting reply to msg 2cd1682479d14cb08b911a7e6a95cfb6 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 723.335259] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2cd1682479d14cb08b911a7e6a95cfb6 [ 723.707947] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-ad36f3a0-38c5-4b94-834f-5999e98f33fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 723.714141] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-ad36f3a0-38c5-4b94-834f-5999e98f33fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 8c92ea76-0de2-41d7-a700-09a95d17ee2a] While scheduling instance, the build request was already deleted. {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1706}} [ 723.732544] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Expecting reply to msg d089ca6da3be44b89e9d48d178095a9d in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 723.849937] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d089ca6da3be44b89e9d48d178095a9d [ 723.856072] nova-conductor[54075]: INFO nova.scheduler.client.report [None req-ad36f3a0-38c5-4b94-834f-5999e98f33fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Deleted allocations for instance 8c92ea76-0de2-41d7-a700-09a95d17ee2a [ 723.860912] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ad36f3a0-38c5-4b94-834f-5999e98f33fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 723.861130] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ad36f3a0-38c5-4b94-834f-5999e98f33fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 723.861295] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ad36f3a0-38c5-4b94-834f-5999e98f33fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 723.861656] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ad36f3a0-38c5-4b94-834f-5999e98f33fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 723.861838] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ad36f3a0-38c5-4b94-834f-5999e98f33fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 723.861994] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ad36f3a0-38c5-4b94-834f-5999e98f33fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 723.869999] nova-conductor[54075]: DEBUG nova.quota [None req-ad36f3a0-38c5-4b94-834f-5999e98f33fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Getting quotas for project aa59611b27bb41beb282e68ccfa6fadc. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 723.872359] nova-conductor[54075]: DEBUG nova.quota [None req-ad36f3a0-38c5-4b94-834f-5999e98f33fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Getting quotas for user ba5969be1a254281b4dffd81aa84ae7e and project aa59611b27bb41beb282e68ccfa6fadc. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 724.235242] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 724.246620] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 724.246886] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 724.247003] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 724.279198] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 724.279461] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 724.279685] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 724.279993] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 724.280214] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 724.280400] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 724.288577] nova-conductor[54075]: DEBUG nova.quota [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Getting quotas for project aa59611b27bb41beb282e68ccfa6fadc. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 724.290609] nova-conductor[54075]: DEBUG nova.quota [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Getting quotas for user ba5969be1a254281b4dffd81aa84ae7e and project aa59611b27bb41beb282e68ccfa6fadc. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 724.295399] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 6163fcd4-cfe4-4432-ba8d-665319fa11ed] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 724.295821] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 724.296066] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 724.296246] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 724.301733] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 6163fcd4-cfe4-4432-ba8d-665319fa11ed] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 724.302380] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 724.302580] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 724.302743] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 724.317966] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 724.318237] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 724.318369] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 726.230317] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] [instance: c9b2ced5-a77c-4bff-b115-ce5c523be630] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b2097d88-fbe6-444e-861d-6f21a30fe2ba, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance c9b2ced5-a77c-4bff-b115-ce5c523be630 was re-scheduled: Binding failed for port b2097d88-fbe6-444e-861d-6f21a30fe2ba, please check neutron logs for more information.\n'] [ 726.230653] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 726.230864] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c9b2ced5-a77c-4bff-b115-ce5c523be630.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c9b2ced5-a77c-4bff-b115-ce5c523be630. [ 726.231086] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] [instance: c9b2ced5-a77c-4bff-b115-ce5c523be630] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c9b2ced5-a77c-4bff-b115-ce5c523be630. [ 726.261316] nova-conductor[54074]: DEBUG nova.network.neutron [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] [instance: c9b2ced5-a77c-4bff-b115-ce5c523be630] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 726.280722] nova-conductor[54074]: DEBUG nova.network.neutron [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] [instance: c9b2ced5-a77c-4bff-b115-ce5c523be630] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 726.284220] nova-conductor[54074]: DEBUG nova.network.neutron [None req-d9ce9fdb-776b-482f-b3a3-b283d6fbdac8 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] [instance: c9b2ced5-a77c-4bff-b115-ce5c523be630] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 730.468922] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=3 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 730.499225] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=9,nova_cell1:SAVEPOINT=2,nova_cell1:INSERT=23,nova_cell1:RELEASE=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 730.509756] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=18,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 730.512035] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=18,nova_api:DELETE=2,nova_api:UPDATE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 733.055029] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] [instance: 26a287d7-4602-4d83-8828-41870a49c343] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b9a39990-885d-406a-a2d2-338706184202, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 26a287d7-4602-4d83-8828-41870a49c343 was re-scheduled: Binding failed for port b9a39990-885d-406a-a2d2-338706184202, please check neutron logs for more information.\n'] [ 733.055676] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 733.055945] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 26a287d7-4602-4d83-8828-41870a49c343.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 26a287d7-4602-4d83-8828-41870a49c343. [ 733.056296] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] [instance: 26a287d7-4602-4d83-8828-41870a49c343] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 26a287d7-4602-4d83-8828-41870a49c343. [ 733.083877] nova-conductor[54075]: DEBUG nova.network.neutron [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] [instance: 26a287d7-4602-4d83-8828-41870a49c343] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 733.100282] nova-conductor[54075]: DEBUG nova.network.neutron [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] [instance: 26a287d7-4602-4d83-8828-41870a49c343] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 733.103930] nova-conductor[54075]: DEBUG nova.network.neutron [None req-8841364b-cbc6-4765-9f39-b253815bd2e0 tempest-ServerRescueNegativeTestJSON-663683599 tempest-ServerRescueNegativeTestJSON-663683599-project-member] [instance: 26a287d7-4602-4d83-8828-41870a49c343] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 734.285237] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=3 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 734.305002] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=7,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=15,nova_cell1:RELEASE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 734.315897] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=19,nova_api:UPDATE=2 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 734.319112] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=14,nova_api:DELETE=3 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 734.973674] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] [instance: dcdb35dd-3c01-4ba6-9a94-9ef3d49caa09] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 89293cc2-155b-47ed-b5ba-f18c8b302e91, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance dcdb35dd-3c01-4ba6-9a94-9ef3d49caa09 was re-scheduled: Binding failed for port 89293cc2-155b-47ed-b5ba-f18c8b302e91, please check neutron logs for more information.\n'] [ 734.973960] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 734.974257] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance dcdb35dd-3c01-4ba6-9a94-9ef3d49caa09.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance dcdb35dd-3c01-4ba6-9a94-9ef3d49caa09. [ 734.974486] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] [instance: dcdb35dd-3c01-4ba6-9a94-9ef3d49caa09] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance dcdb35dd-3c01-4ba6-9a94-9ef3d49caa09. [ 735.005026] nova-conductor[54074]: DEBUG nova.network.neutron [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] [instance: dcdb35dd-3c01-4ba6-9a94-9ef3d49caa09] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 735.020425] nova-conductor[54074]: DEBUG nova.network.neutron [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] [instance: dcdb35dd-3c01-4ba6-9a94-9ef3d49caa09] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 735.024145] nova-conductor[54074]: DEBUG nova.network.neutron [None req-a5b9c9eb-e624-4ab4-9dd9-440374e69d57 tempest-ServerAddressesTestJSON-1380179577 tempest-ServerAddressesTestJSON-1380179577-project-member] [instance: dcdb35dd-3c01-4ba6-9a94-9ef3d49caa09] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 740.748747] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Expecting reply to msg e208fc32e74f42c7bf656efa910ac7de in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 740.892131] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e208fc32e74f42c7bf656efa910ac7de [ 741.251810] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 741.266646] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 741.266882] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 741.267049] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 741.323951] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 741.324205] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 741.324825] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 741.325196] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 741.325578] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 741.325757] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 741.335172] nova-conductor[54075]: DEBUG nova.quota [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Getting quotas for project 68c6abcd904e419c828b564f5cd78bc5. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 741.338131] nova-conductor[54075]: DEBUG nova.quota [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Getting quotas for user 26e0d6bc5c7b451abae6add4088b1c62 and project 68c6abcd904e419c828b564f5cd78bc5. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 741.344671] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] [instance: 252b7e84-4f91-4078-a81c-392d622b6ce2] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 741.345083] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 741.345272] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 741.345429] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 741.360783] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] [instance: 252b7e84-4f91-4078-a81c-392d622b6ce2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 741.361486] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 741.361696] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 741.361863] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 741.387233] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 741.387471] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 741.387641] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 742.106471] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] [instance: 29715a53-7a71-4708-b522-e678fe5bd6a9] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port a59f3c1f-3c1e-42a5-9305-85d22fee72e2, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 29715a53-7a71-4708-b522-e678fe5bd6a9 was re-scheduled: Binding failed for port a59f3c1f-3c1e-42a5-9305-85d22fee72e2, please check neutron logs for more information.\n'] [ 742.107839] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 742.108180] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 29715a53-7a71-4708-b522-e678fe5bd6a9.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 29715a53-7a71-4708-b522-e678fe5bd6a9. [ 742.108516] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] [instance: 29715a53-7a71-4708-b522-e678fe5bd6a9] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 29715a53-7a71-4708-b522-e678fe5bd6a9. [ 742.159592] nova-conductor[54075]: DEBUG nova.network.neutron [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] [instance: 29715a53-7a71-4708-b522-e678fe5bd6a9] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 742.185186] nova-conductor[54075]: DEBUG nova.network.neutron [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] [instance: 29715a53-7a71-4708-b522-e678fe5bd6a9] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 742.189972] nova-conductor[54075]: DEBUG nova.network.neutron [None req-a8f8d7f0-f1b8-41ca-b637-5158f1da0102 tempest-ServerActionsTestOtherB-638853083 tempest-ServerActionsTestOtherB-638853083-project-member] [instance: 29715a53-7a71-4708-b522-e678fe5bd6a9] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 746.708846] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 446bd52c-4ffb-4e77-89fb-3e8535ceb4af] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 7dc696ee-8e41-4b30-9400-d4ba7c26b170, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 446bd52c-4ffb-4e77-89fb-3e8535ceb4af was re-scheduled: Binding failed for port 7dc696ee-8e41-4b30-9400-d4ba7c26b170, please check neutron logs for more information.\n'] [ 746.709424] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 746.709657] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 446bd52c-4ffb-4e77-89fb-3e8535ceb4af.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 446bd52c-4ffb-4e77-89fb-3e8535ceb4af. [ 746.709868] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 446bd52c-4ffb-4e77-89fb-3e8535ceb4af] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 446bd52c-4ffb-4e77-89fb-3e8535ceb4af. [ 746.744962] nova-conductor[54074]: DEBUG nova.network.neutron [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 446bd52c-4ffb-4e77-89fb-3e8535ceb4af] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 746.761376] nova-conductor[54074]: DEBUG nova.network.neutron [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 446bd52c-4ffb-4e77-89fb-3e8535ceb4af] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 746.765232] nova-conductor[54074]: DEBUG nova.network.neutron [None req-f13d282e-55e7-4a27-be61-3cbafeaaa1aa tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 446bd52c-4ffb-4e77-89fb-3e8535ceb4af] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 747.841752] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Expecting reply to msg 1d28bc50b6b24f3e986264a5e084512c in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 747.985415] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1d28bc50b6b24f3e986264a5e084512c [ 748.344647] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 748.358611] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.358855] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.359034] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 748.387222] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.387455] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.387628] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 748.388050] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.388248] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.388416] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 748.398532] nova-conductor[54075]: DEBUG nova.quota [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Getting quotas for project 423f777bec3c474a91970fce3e308097. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 748.401459] nova-conductor[54075]: DEBUG nova.quota [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Getting quotas for user e4ea22bb21004f69a2b27d306493db45 and project 423f777bec3c474a91970fce3e308097. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 748.407410] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 3c0ed5ff-4232-4ab5-a91c-d82da0008f8e] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 748.407750] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.407985] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.408209] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 748.418822] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 3c0ed5ff-4232-4ab5-a91c-d82da0008f8e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 748.419653] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.419874] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.420145] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 748.445062] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.445503] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.445503] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 751.217038] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] [instance: 6f8e35f3-4b35-449c-9e60-1e0624f41cd2] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 2f48948e-41a3-4548-a9be-24230ffff871, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 6f8e35f3-4b35-449c-9e60-1e0624f41cd2 was re-scheduled: Binding failed for port 2f48948e-41a3-4548-a9be-24230ffff871, please check neutron logs for more information.\n'] [ 751.217948] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 751.218178] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6f8e35f3-4b35-449c-9e60-1e0624f41cd2.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6f8e35f3-4b35-449c-9e60-1e0624f41cd2. [ 751.218392] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] [instance: 6f8e35f3-4b35-449c-9e60-1e0624f41cd2] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6f8e35f3-4b35-449c-9e60-1e0624f41cd2. [ 751.238543] nova-conductor[54074]: DEBUG nova.network.neutron [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] [instance: 6f8e35f3-4b35-449c-9e60-1e0624f41cd2] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 751.253264] nova-conductor[54074]: DEBUG nova.network.neutron [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] [instance: 6f8e35f3-4b35-449c-9e60-1e0624f41cd2] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 751.256500] nova-conductor[54074]: DEBUG nova.network.neutron [None req-33e81f1b-3607-4a32-9744-456196feb4a1 tempest-ServerDiagnosticsTest-1641091972 tempest-ServerDiagnosticsTest-1641091972-project-member] [instance: 6f8e35f3-4b35-449c-9e60-1e0624f41cd2] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 753.129078] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: c9a6d28b-52f8-4636-886a-c74f0900e761] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 7b566737-b743-4fbb-a685-3882ecc657df, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance c9a6d28b-52f8-4636-886a-c74f0900e761 was re-scheduled: Binding failed for port 7b566737-b743-4fbb-a685-3882ecc657df, please check neutron logs for more information.\n'] [ 753.129699] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 753.129936] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c9a6d28b-52f8-4636-886a-c74f0900e761.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c9a6d28b-52f8-4636-886a-c74f0900e761. [ 753.130252] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: c9a6d28b-52f8-4636-886a-c74f0900e761] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c9a6d28b-52f8-4636-886a-c74f0900e761. [ 753.170084] nova-conductor[54075]: DEBUG nova.network.neutron [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: c9a6d28b-52f8-4636-886a-c74f0900e761] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 753.186115] nova-conductor[54075]: DEBUG nova.network.neutron [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: c9a6d28b-52f8-4636-886a-c74f0900e761] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 753.189458] nova-conductor[54075]: DEBUG nova.network.neutron [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: c9a6d28b-52f8-4636-886a-c74f0900e761] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 758.393735] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=2 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 758.422458] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=6,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=15,nova_cell1:RELEASE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 758.432067] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=14,nova_api:UPDATE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 758.434395] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=10,nova_api:UPDATE=1,nova_api:DELETE=2 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 760.148884] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: 27b21d98-bf4d-47ee-84e8-d5a3d10d87a4] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port c2098700-1246-4d9f-9eff-78376ba87e82, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 27b21d98-bf4d-47ee-84e8-d5a3d10d87a4 was re-scheduled: Binding failed for port c2098700-1246-4d9f-9eff-78376ba87e82, please check neutron logs for more information.\n'] [ 760.149208] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 760.149452] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 27b21d98-bf4d-47ee-84e8-d5a3d10d87a4.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 27b21d98-bf4d-47ee-84e8-d5a3d10d87a4. [ 760.149628] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: 27b21d98-bf4d-47ee-84e8-d5a3d10d87a4] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 27b21d98-bf4d-47ee-84e8-d5a3d10d87a4. [ 760.201848] nova-conductor[54074]: DEBUG nova.network.neutron [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: 27b21d98-bf4d-47ee-84e8-d5a3d10d87a4] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 760.222695] nova-conductor[54074]: DEBUG nova.network.neutron [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: 27b21d98-bf4d-47ee-84e8-d5a3d10d87a4] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 760.231145] nova-conductor[54074]: DEBUG nova.network.neutron [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: 27b21d98-bf4d-47ee-84e8-d5a3d10d87a4] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 762.023631] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: 9f77d364-928f-4595-9253-8bb216b9215b] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 2051acbf-f948-4fb9-b38a-44883022ff72, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 9f77d364-928f-4595-9253-8bb216b9215b was re-scheduled: Binding failed for port 2051acbf-f948-4fb9-b38a-44883022ff72, please check neutron logs for more information.\n'] [ 762.024459] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 762.024546] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9f77d364-928f-4595-9253-8bb216b9215b.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9f77d364-928f-4595-9253-8bb216b9215b. [ 762.024758] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: 9f77d364-928f-4595-9253-8bb216b9215b] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9f77d364-928f-4595-9253-8bb216b9215b. [ 762.047604] nova-conductor[54075]: DEBUG nova.network.neutron [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: 9f77d364-928f-4595-9253-8bb216b9215b] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 762.069691] nova-conductor[54075]: DEBUG nova.network.neutron [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: 9f77d364-928f-4595-9253-8bb216b9215b] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 762.073614] nova-conductor[54075]: DEBUG nova.network.neutron [None req-5174db81-624c-4ad9-99ed-ccc1b5918206 tempest-ListServersNegativeTestJSON-505733123 tempest-ListServersNegativeTestJSON-505733123-project-member] [instance: 9f77d364-928f-4595-9253-8bb216b9215b] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 763.628695] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=393,nova_cell1:SAVEPOINT=35,nova_cell1:RELEASE=35,nova_cell1:UPDATE=191,nova_cell1:INSERT=30 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 767.915994] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=329,nova_cell1:UPDATE=156,nova_cell1:INSERT=21,nova_cell1:SAVEPOINT=42,nova_cell1:RELEASE=42 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 776.326782] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Expecting reply to msg ce7b46e1271445f7b67b3b0784bd042b in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 776.458271] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ce7b46e1271445f7b67b3b0784bd042b [ 776.829792] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 776.844868] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 776.845252] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 776.845552] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 776.882256] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 776.882623] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 776.882893] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 776.883325] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 776.883507] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 776.883665] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 776.893243] nova-conductor[54074]: DEBUG nova.quota [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Getting quotas for project 8a7b7a7806e84f1ebca2207ff836ce46. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 776.896193] nova-conductor[54074]: DEBUG nova.quota [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Getting quotas for user 20e2b6e1fa34439587b2b0fc42646c1e and project 8a7b7a7806e84f1ebca2207ff836ce46. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 776.901034] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] [instance: f453b695-8abd-44fa-8468-75c6aaeec19a] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 776.901466] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 776.901670] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 776.901839] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 776.906404] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] [instance: f453b695-8abd-44fa-8468-75c6aaeec19a] block_device_mapping [BlockDeviceMapping(attachment_id=075519bd-337c-41bc-86d2-2aba2978da8d,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='9435a93d-7799-4ec6-91c6-ecbe7ea15abc',volume_size=1,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 776.907052] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 776.907252] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 776.907457] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 776.920948] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 776.921166] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 776.921333] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 778.915053] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 832c9ce1-6344-485a-a9ef-6950d1c78ef9] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port bd3d8b29-3381-4851-99c3-877677b6056f, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 832c9ce1-6344-485a-a9ef-6950d1c78ef9 was re-scheduled: Binding failed for port bd3d8b29-3381-4851-99c3-877677b6056f, please check neutron logs for more information.\n'] [ 778.915593] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 778.915846] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 832c9ce1-6344-485a-a9ef-6950d1c78ef9.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 832c9ce1-6344-485a-a9ef-6950d1c78ef9. [ 778.916055] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 832c9ce1-6344-485a-a9ef-6950d1c78ef9] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 832c9ce1-6344-485a-a9ef-6950d1c78ef9. [ 778.944985] nova-conductor[54075]: DEBUG nova.network.neutron [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 832c9ce1-6344-485a-a9ef-6950d1c78ef9] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 778.962400] nova-conductor[54075]: DEBUG nova.network.neutron [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 832c9ce1-6344-485a-a9ef-6950d1c78ef9] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 778.965911] nova-conductor[54075]: DEBUG nova.network.neutron [None req-95852330-b22c-4e57-b3fc-19c0b2a26db1 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 832c9ce1-6344-485a-a9ef-6950d1c78ef9] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 779.725640] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Expecting reply to msg a9286fb566d149899294c616cdc112ae in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 779.872811] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a9286fb566d149899294c616cdc112ae [ 780.228139] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 780.242367] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 780.242679] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 780.242913] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 780.273962] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 780.274295] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 780.274527] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 780.274984] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 780.275226] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 780.275451] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 780.289228] nova-conductor[54074]: DEBUG nova.quota [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Getting quotas for project 71a669cc62964b05bdaa71442c08a566. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 780.292184] nova-conductor[54074]: DEBUG nova.quota [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Getting quotas for user 2d003a5943d141cc9165e43148dec381 and project 71a669cc62964b05bdaa71442c08a566. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 780.301152] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: e2e09174-6ba1-44ad-ba3e-cdcae5a2d698] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 780.301726] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 780.302001] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 780.302235] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 780.305565] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: e2e09174-6ba1-44ad-ba3e-cdcae5a2d698] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 780.306303] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 780.306572] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 780.306822] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 780.325655] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 780.325991] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 780.326243] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 783.235865] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] [instance: 3d99c7df-b031-4187-988c-f642f79073d3] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port d19cd207-7c44-4737-9b63-dd316aa946f5, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 3d99c7df-b031-4187-988c-f642f79073d3 was re-scheduled: Binding failed for port d19cd207-7c44-4737-9b63-dd316aa946f5, please check neutron logs for more information.\n'] [ 783.240481] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 783.240825] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3d99c7df-b031-4187-988c-f642f79073d3.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3d99c7df-b031-4187-988c-f642f79073d3. [ 783.241223] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] [instance: 3d99c7df-b031-4187-988c-f642f79073d3] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3d99c7df-b031-4187-988c-f642f79073d3. [ 783.270319] nova-conductor[54074]: DEBUG nova.network.neutron [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] [instance: 3d99c7df-b031-4187-988c-f642f79073d3] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 783.286365] nova-conductor[54074]: DEBUG nova.network.neutron [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] [instance: 3d99c7df-b031-4187-988c-f642f79073d3] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 783.289542] nova-conductor[54074]: DEBUG nova.network.neutron [None req-bd7c0011-a90a-453e-881d-fe8dbfc17285 tempest-ServersV294TestFqdnHostnames-815487066 tempest-ServersV294TestFqdnHostnames-815487066-project-member] [instance: 3d99c7df-b031-4187-988c-f642f79073d3] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 790.010928] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] [instance: 87304cf6-e65f-41de-ab6f-d2170aaa9064] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 1c6a3483-455d-4776-a8a1-23760212d4ef, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 87304cf6-e65f-41de-ab6f-d2170aaa9064 was re-scheduled: Binding failed for port 1c6a3483-455d-4776-a8a1-23760212d4ef, please check neutron logs for more information.\n'] [ 790.012033] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 790.012033] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 87304cf6-e65f-41de-ab6f-d2170aaa9064.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 87304cf6-e65f-41de-ab6f-d2170aaa9064. [ 790.012181] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] [instance: 87304cf6-e65f-41de-ab6f-d2170aaa9064] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 87304cf6-e65f-41de-ab6f-d2170aaa9064. [ 790.038469] nova-conductor[54075]: DEBUG nova.network.neutron [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] [instance: 87304cf6-e65f-41de-ab6f-d2170aaa9064] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 790.054127] nova-conductor[54075]: DEBUG nova.network.neutron [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] [instance: 87304cf6-e65f-41de-ab6f-d2170aaa9064] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 790.056904] nova-conductor[54075]: DEBUG nova.network.neutron [None req-91380fdb-bfd1-40aa-a541-58dca23365be tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] [instance: 87304cf6-e65f-41de-ab6f-d2170aaa9064] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 790.283541] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 790.308390] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=6,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=16,nova_cell1:RELEASE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 790.317418] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=15 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 790.320519] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=2,nova_api:DELETE=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 791.905234] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] [instance: 90c50f92-c1ff-4ac9-a819-ae0083884e28] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 27d29d58-7775-4338-8153-6267d4a560a3, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 90c50f92-c1ff-4ac9-a819-ae0083884e28 was re-scheduled: Binding failed for port 27d29d58-7775-4338-8153-6267d4a560a3, please check neutron logs for more information.\n'] [ 791.905812] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 791.906089] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 90c50f92-c1ff-4ac9-a819-ae0083884e28.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 90c50f92-c1ff-4ac9-a819-ae0083884e28. [ 791.906329] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] [instance: 90c50f92-c1ff-4ac9-a819-ae0083884e28] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 90c50f92-c1ff-4ac9-a819-ae0083884e28. [ 791.926312] nova-conductor[54074]: DEBUG nova.network.neutron [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] [instance: 90c50f92-c1ff-4ac9-a819-ae0083884e28] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 791.946998] nova-conductor[54074]: DEBUG nova.network.neutron [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] [instance: 90c50f92-c1ff-4ac9-a819-ae0083884e28] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 791.950359] nova-conductor[54074]: DEBUG nova.network.neutron [None req-cbde3920-e9fb-4541-a32b-dc71604200bd tempest-ListImageFiltersTestJSON-1730796357 tempest-ListImageFiltersTestJSON-1730796357-project-member] [instance: 90c50f92-c1ff-4ac9-a819-ae0083884e28] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 798.569646] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] [instance: ae026dca-dc05-4710-8a03-4e792a0dc61d] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b2910613-b431-4ef8-9ab4-20a3b9638084, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ae026dca-dc05-4710-8a03-4e792a0dc61d was re-scheduled: Binding failed for port b2910613-b431-4ef8-9ab4-20a3b9638084, please check neutron logs for more information.\n'] [ 798.570174] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 798.570434] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ae026dca-dc05-4710-8a03-4e792a0dc61d.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ae026dca-dc05-4710-8a03-4e792a0dc61d. [ 798.570652] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] [instance: ae026dca-dc05-4710-8a03-4e792a0dc61d] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ae026dca-dc05-4710-8a03-4e792a0dc61d. [ 798.606026] nova-conductor[54074]: DEBUG nova.network.neutron [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] [instance: ae026dca-dc05-4710-8a03-4e792a0dc61d] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 798.621804] nova-conductor[54074]: DEBUG nova.network.neutron [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] [instance: ae026dca-dc05-4710-8a03-4e792a0dc61d] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 798.625068] nova-conductor[54074]: DEBUG nova.network.neutron [None req-ba02dd96-7ea9-4b4a-904a-ab5cbe3d08b1 tempest-AttachInterfacesUnderV243Test-451709166 tempest-AttachInterfacesUnderV243Test-451709166-project-member] [instance: ae026dca-dc05-4710-8a03-4e792a0dc61d] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 802.888668] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] [instance: 16b04a1b-0ab3-4386-a1eb-74ef3e46a553] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port f735d74f-e61d-4468-b208-318e406dcc17, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 16b04a1b-0ab3-4386-a1eb-74ef3e46a553 was re-scheduled: Binding failed for port f735d74f-e61d-4468-b208-318e406dcc17, please check neutron logs for more information.\n'] [ 802.889076] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 802.889645] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 16b04a1b-0ab3-4386-a1eb-74ef3e46a553.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 16b04a1b-0ab3-4386-a1eb-74ef3e46a553. [ 802.890069] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] [instance: 16b04a1b-0ab3-4386-a1eb-74ef3e46a553] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 16b04a1b-0ab3-4386-a1eb-74ef3e46a553. [ 802.910279] nova-conductor[54075]: DEBUG nova.network.neutron [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] [instance: 16b04a1b-0ab3-4386-a1eb-74ef3e46a553] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 802.924653] nova-conductor[54075]: DEBUG nova.network.neutron [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] [instance: 16b04a1b-0ab3-4386-a1eb-74ef3e46a553] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 802.928744] nova-conductor[54075]: DEBUG nova.network.neutron [None req-a726516d-0952-438f-baf7-b9ad271fcf78 tempest-MigrationsAdminTest-670983471 tempest-MigrationsAdminTest-670983471-project-member] [instance: 16b04a1b-0ab3-4386-a1eb-74ef3e46a553] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 805.580325] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Expecting reply to msg 008b6725cfea4c13a1366f401d0dc83f in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 805.711825] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 008b6725cfea4c13a1366f401d0dc83f [ 806.086378] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 806.106508] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 806.106773] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 806.106887] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 806.153465] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 806.153699] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 806.153871] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 806.154237] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 806.154418] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 806.154576] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 806.164705] nova-conductor[54075]: DEBUG nova.quota [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Getting quotas for project dd6c831cfae3480c9ebea53f615c811e. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 806.166923] nova-conductor[54075]: DEBUG nova.quota [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Getting quotas for user 792f3e11b5ab49ab95b8d0344604ae82 and project dd6c831cfae3480c9ebea53f615c811e. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 806.171962] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] [instance: 309a7bae-82f5-4b9e-ac86-e0f1803f2585] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 806.172451] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 806.172660] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 806.172822] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 806.178202] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] [instance: 309a7bae-82f5-4b9e-ac86-e0f1803f2585] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 806.178827] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 806.179040] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 806.179183] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 806.194544] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 806.194743] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 806.194906] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d6cec8f9-58e6-4c1c-ac20-468cef5a3ec1 tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 807.441259] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] [instance: 08638aac-2c6c-4580-9894-6b3b3c1ec484] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port ce1aee7f-24df-45b4-acff-b0bbb0657cbe, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 08638aac-2c6c-4580-9894-6b3b3c1ec484 was re-scheduled: Binding failed for port ce1aee7f-24df-45b4-acff-b0bbb0657cbe, please check neutron logs for more information.\n'] [ 807.441836] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 807.442054] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 08638aac-2c6c-4580-9894-6b3b3c1ec484.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 08638aac-2c6c-4580-9894-6b3b3c1ec484. [ 807.442256] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] [instance: 08638aac-2c6c-4580-9894-6b3b3c1ec484] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 08638aac-2c6c-4580-9894-6b3b3c1ec484. [ 807.469523] nova-conductor[54074]: DEBUG nova.network.neutron [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] [instance: 08638aac-2c6c-4580-9894-6b3b3c1ec484] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 807.490776] nova-conductor[54074]: DEBUG nova.network.neutron [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] [instance: 08638aac-2c6c-4580-9894-6b3b3c1ec484] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 807.505887] nova-conductor[54074]: DEBUG nova.network.neutron [None req-101896f4-f52c-4a4f-94f6-1fd61758c65f tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] [instance: 08638aac-2c6c-4580-9894-6b3b3c1ec484] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 809.296293] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] [instance: 66a0a424-ecb6-43df-9b47-946ff1e1b7b2] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 9e03f450-e277-4f0a-98c6-050a2f8f2359, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 66a0a424-ecb6-43df-9b47-946ff1e1b7b2 was re-scheduled: Binding failed for port 9e03f450-e277-4f0a-98c6-050a2f8f2359, please check neutron logs for more information.\n'] [ 809.296887] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 809.297118] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 66a0a424-ecb6-43df-9b47-946ff1e1b7b2.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 66a0a424-ecb6-43df-9b47-946ff1e1b7b2. [ 809.297340] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] [instance: 66a0a424-ecb6-43df-9b47-946ff1e1b7b2] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 66a0a424-ecb6-43df-9b47-946ff1e1b7b2. [ 809.340213] nova-conductor[54074]: DEBUG nova.network.neutron [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] [instance: 66a0a424-ecb6-43df-9b47-946ff1e1b7b2] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 809.363897] nova-conductor[54074]: DEBUG nova.network.neutron [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] [instance: 66a0a424-ecb6-43df-9b47-946ff1e1b7b2] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 809.367319] nova-conductor[54074]: DEBUG nova.network.neutron [None req-aa878a52-ac9d-4ae5-8cc8-b3ed9736f7af tempest-SecurityGroupsTestJSON-704559567 tempest-SecurityGroupsTestJSON-704559567-project-member] [instance: 66a0a424-ecb6-43df-9b47-946ff1e1b7b2] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 810.063411] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Expecting reply to msg ebb42b8b9e4f42edad7582e10cba70ce in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 810.218055] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ebb42b8b9e4f42edad7582e10cba70ce [ 810.570515] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 810.582735] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 810.582956] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 810.583125] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 810.614758] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 810.614758] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 810.614758] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 810.614758] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 810.614758] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 810.614758] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 810.635466] nova-conductor[54074]: DEBUG nova.quota [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Getting quotas for project 8d361e27776e47eeadaa4a29b4f9338f. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 810.640260] nova-conductor[54074]: DEBUG nova.quota [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Getting quotas for user 0933891c54584b059f68770a8c930f1d and project 8d361e27776e47eeadaa4a29b4f9338f. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 810.647631] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] [instance: 3f99ec88-f05f-4583-b08b-d40fb37e275e] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 810.647631] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 810.648065] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 810.648065] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 810.653340] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] [instance: 3f99ec88-f05f-4583-b08b-d40fb37e275e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 810.653670] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 810.653670] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 810.654282] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 810.669449] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 810.669761] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 810.670034] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 815.911987] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Expecting reply to msg ee2c5f73b72841868482125337790a77 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 816.135132] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ee2c5f73b72841868482125337790a77 [ 816.159670] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 816.181573] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=3,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=8,nova_cell1:RELEASE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 816.193309] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=6,nova_api:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 816.416061] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Took 0.51 seconds to select destinations for 2 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 816.431565] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 816.431786] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 816.431953] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 816.465689] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 816.465921] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 816.466087] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 816.491466] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 816.491693] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 816.491853] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 816.492254] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 816.492429] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 816.492580] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 816.502282] nova-conductor[54075]: DEBUG nova.quota [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Getting quotas for project 4e10657828d9480b948e59b98490572b. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 816.505087] nova-conductor[54075]: DEBUG nova.quota [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Getting quotas for user feae790f4343445c86cfb1b39cb9636e and project 4e10657828d9480b948e59b98490572b. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 816.510628] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 1a38b70f-eabe-4b11-a371-cf971184211f] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 816.511096] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 816.511298] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 816.511466] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 816.516210] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 1a38b70f-eabe-4b11-a371-cf971184211f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 816.516840] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 816.517040] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 816.517203] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 816.531149] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 816.531369] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 816.531532] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 816.537651] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 8a6d10c8-bd2b-40dd-9897-8f30223abe81] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 816.538080] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 816.538290] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 816.538472] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 816.541033] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 8a6d10c8-bd2b-40dd-9897-8f30223abe81] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 816.541622] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 816.541812] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 816.541973] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 816.560475] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 816.560699] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 816.560865] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 820.623391] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 820.655776] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=3,nova_cell1:INSERT=7 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 820.663981] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=7 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 820.666006] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=5,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 823.975050] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SAVEPOINT=25,nova_cell1:RELEASE=25,nova_cell1:SELECT=276,nova_cell1:UPDATE=140,nova_cell1:INSERT=18,nova_cell1:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 825.304071] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: e8c77459-e3a3-4a68-9f76-0757dd0f2587] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port a1df0846-1c39-4992-9a51-d7ac2e394a3c, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance e8c77459-e3a3-4a68-9f76-0757dd0f2587 was re-scheduled: Binding failed for port a1df0846-1c39-4992-9a51-d7ac2e394a3c, please check neutron logs for more information.\n'] [ 825.305005] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 825.305349] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e8c77459-e3a3-4a68-9f76-0757dd0f2587.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e8c77459-e3a3-4a68-9f76-0757dd0f2587. [ 825.305803] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: e8c77459-e3a3-4a68-9f76-0757dd0f2587] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e8c77459-e3a3-4a68-9f76-0757dd0f2587. [ 825.327741] nova-conductor[54075]: DEBUG nova.network.neutron [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: e8c77459-e3a3-4a68-9f76-0757dd0f2587] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 825.342776] nova-conductor[54075]: DEBUG nova.network.neutron [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: e8c77459-e3a3-4a68-9f76-0757dd0f2587] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 825.346120] nova-conductor[54075]: DEBUG nova.network.neutron [None req-3d82fb94-09c2-4954-8f91-55aeb880b429 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: e8c77459-e3a3-4a68-9f76-0757dd0f2587] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 826.497559] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 826.546441] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=5,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=15,nova_cell1:RELEASE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 826.550108] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=15,nova_api:UPDATE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 826.558647] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=7,nova_api:UPDATE=2,nova_api:DELETE=2 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 827.128654] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: 9a1c4327-64b3-4c4d-b6ae-77959084b405] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port f971e1a0-a370-44fc-acd0-66f2e04e3b60, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 9a1c4327-64b3-4c4d-b6ae-77959084b405 was re-scheduled: Binding failed for port f971e1a0-a370-44fc-acd0-66f2e04e3b60, please check neutron logs for more information.\n'] [ 827.129625] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 827.129864] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9a1c4327-64b3-4c4d-b6ae-77959084b405.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9a1c4327-64b3-4c4d-b6ae-77959084b405. [ 827.130072] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: 9a1c4327-64b3-4c4d-b6ae-77959084b405] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9a1c4327-64b3-4c4d-b6ae-77959084b405. [ 827.148605] nova-conductor[54075]: DEBUG nova.network.neutron [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: 9a1c4327-64b3-4c4d-b6ae-77959084b405] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 827.163852] nova-conductor[54075]: DEBUG nova.network.neutron [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: 9a1c4327-64b3-4c4d-b6ae-77959084b405] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 827.166810] nova-conductor[54075]: DEBUG nova.network.neutron [None req-549cd6aa-84cb-440a-80fe-04e39bab5060 tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: 9a1c4327-64b3-4c4d-b6ae-77959084b405] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 828.135433] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=288,nova_cell1:UPDATE=124,nova_cell1:INSERT=19,nova_cell1:SAVEPOINT=28,nova_cell1:RELEASE=28 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 831.303099] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: a6ec5486-0843-4c38-b187-35d5296965a7] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port e8178202-8899-4251-9c65-7ab8b9217948, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance a6ec5486-0843-4c38-b187-35d5296965a7 was re-scheduled: Binding failed for port e8178202-8899-4251-9c65-7ab8b9217948, please check neutron logs for more information.\n'] [ 831.303914] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 831.304344] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a6ec5486-0843-4c38-b187-35d5296965a7.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a6ec5486-0843-4c38-b187-35d5296965a7. [ 831.304814] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: a6ec5486-0843-4c38-b187-35d5296965a7] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a6ec5486-0843-4c38-b187-35d5296965a7. [ 831.328032] nova-conductor[54074]: DEBUG nova.network.neutron [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: a6ec5486-0843-4c38-b187-35d5296965a7] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 831.359738] nova-conductor[54074]: DEBUG nova.network.neutron [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: a6ec5486-0843-4c38-b187-35d5296965a7] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 831.363546] nova-conductor[54074]: DEBUG nova.network.neutron [None req-da75bf2d-3b72-4d73-ab8c-b9ac523ffddf tempest-ListServerFiltersTestJSON-1171420207 tempest-ListServerFiltersTestJSON-1171420207-project-member] [instance: a6ec5486-0843-4c38-b187-35d5296965a7] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 834.331546] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-d3984e34-1f38-49e7-8018-ddd55561e9fe tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Targeting cell 89980bcd-ea65-4338-957c-d9d250569837(cell1) for conductor method rebuild_instance {{(pid=54074) wrapper /opt/stack/nova/nova/conductor/manager.py:95}} [ 834.331929] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d3984e34-1f38-49e7-8018-ddd55561e9fe tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 834.332050] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d3984e34-1f38-49e7-8018-ddd55561e9fe tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 834.332257] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-d3984e34-1f38-49e7-8018-ddd55561e9fe tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 834.343669] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-d3984e34-1f38-49e7-8018-ddd55561e9fe tempest-ServerShowV247Test-1781566314 tempest-ServerShowV247Test-1781566314-project-member] [instance: 900e1e1e-5635-4782-bd87-046dd2af7dad] No migration record for the rebuild/evacuate request. {{(pid=54074) rebuild_instance /opt/stack/nova/nova/conductor/manager.py:1227}} [ 835.509402] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] [instance: c7ec619c-1b00-4d58-a593-671c0139c4e3] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port bf7efc6a-fe14-4763-9bd8-3116fd5d40e2, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance c7ec619c-1b00-4d58-a593-671c0139c4e3 was re-scheduled: Binding failed for port bf7efc6a-fe14-4763-9bd8-3116fd5d40e2, please check neutron logs for more information.\n'] [ 835.509402] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 835.509784] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c7ec619c-1b00-4d58-a593-671c0139c4e3.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c7ec619c-1b00-4d58-a593-671c0139c4e3. [ 835.510909] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] [instance: c7ec619c-1b00-4d58-a593-671c0139c4e3] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c7ec619c-1b00-4d58-a593-671c0139c4e3. [ 835.541084] nova-conductor[54074]: DEBUG nova.network.neutron [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] [instance: c7ec619c-1b00-4d58-a593-671c0139c4e3] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 835.564117] nova-conductor[54074]: DEBUG nova.network.neutron [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] [instance: c7ec619c-1b00-4d58-a593-671c0139c4e3] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 835.565236] nova-conductor[54074]: DEBUG nova.network.neutron [None req-6c806be6-d3fa-4acb-a2ea-b681958510ea tempest-ServersTestJSON-1195313588 tempest-ServersTestJSON-1195313588-project-member] [instance: c7ec619c-1b00-4d58-a593-671c0139c4e3] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 842.029885] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] [instance: a04d014e-bed6-4e4b-a5eb-316d88c174f0] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port bb0cdfe1-f3b4-46b1-a6e1-1170d0541113, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance a04d014e-bed6-4e4b-a5eb-316d88c174f0 was re-scheduled: Binding failed for port bb0cdfe1-f3b4-46b1-a6e1-1170d0541113, please check neutron logs for more information.\n'] [ 842.030434] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 842.030669] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a04d014e-bed6-4e4b-a5eb-316d88c174f0.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a04d014e-bed6-4e4b-a5eb-316d88c174f0. [ 842.030934] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] [instance: a04d014e-bed6-4e4b-a5eb-316d88c174f0] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a04d014e-bed6-4e4b-a5eb-316d88c174f0. [ 842.067359] nova-conductor[54075]: DEBUG nova.network.neutron [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] [instance: a04d014e-bed6-4e4b-a5eb-316d88c174f0] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 842.089252] nova-conductor[54075]: DEBUG nova.network.neutron [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] [instance: a04d014e-bed6-4e4b-a5eb-316d88c174f0] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 842.092790] nova-conductor[54075]: DEBUG nova.network.neutron [None req-39c2b136-fb3e-44f7-ac09-088b8f850b73 tempest-AttachVolumeShelveTestJSON-1463436007 tempest-AttachVolumeShelveTestJSON-1463436007-project-member] [instance: a04d014e-bed6-4e4b-a5eb-316d88c174f0] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 843.976937] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Expecting reply to msg b8bf64d141f04a949139bf230c7b50bc in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 844.166804] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b8bf64d141f04a949139bf230c7b50bc [ 844.330313] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 844.350009] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 844.367213] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=4,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 844.479393] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 844.498095] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 844.498319] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 844.498497] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 844.537020] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 844.537255] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 844.537417] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 844.537787] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 844.537987] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 844.538152] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 844.546372] nova-conductor[54075]: DEBUG nova.quota [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Getting quotas for project 382a0ffa122c435cac5fd7a5e309a08d. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 844.549133] nova-conductor[54075]: DEBUG nova.quota [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Getting quotas for user 0c52ce34c6a5436b862e2d0d2732c59a and project 382a0ffa122c435cac5fd7a5e309a08d. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 844.559349] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] [instance: a276656a-67b0-4ceb-918f-cfb323ed09fd] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 844.559795] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 844.560013] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 844.560193] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 844.567896] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] [instance: a276656a-67b0-4ceb-918f-cfb323ed09fd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 844.568629] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 844.568839] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 844.569010] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 844.585025] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 844.585249] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 844.585398] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 854.541942] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 854.570967] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=3,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=8,nova_cell1:RELEASE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 854.580941] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=7 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 854.582323] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=5,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 856.330555] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] [instance: afc5587e-7fd5-4b07-aff8-98ef8358985f] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 2be45127-dad8-4c66-98a1-3f9c554f3a2e, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance afc5587e-7fd5-4b07-aff8-98ef8358985f was re-scheduled: Binding failed for port 2be45127-dad8-4c66-98a1-3f9c554f3a2e, please check neutron logs for more information.\n'] [ 856.331141] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 856.331365] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance afc5587e-7fd5-4b07-aff8-98ef8358985f.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance afc5587e-7fd5-4b07-aff8-98ef8358985f. [ 856.331573] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] [instance: afc5587e-7fd5-4b07-aff8-98ef8358985f] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance afc5587e-7fd5-4b07-aff8-98ef8358985f. [ 856.360722] nova-conductor[54074]: DEBUG nova.network.neutron [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] [instance: afc5587e-7fd5-4b07-aff8-98ef8358985f] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 856.375627] nova-conductor[54074]: DEBUG nova.network.neutron [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] [instance: afc5587e-7fd5-4b07-aff8-98ef8358985f] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 856.379445] nova-conductor[54074]: DEBUG nova.network.neutron [None req-f54d10c6-4e90-4c0d-be44-3717a1211a85 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] [instance: afc5587e-7fd5-4b07-aff8-98ef8358985f] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 859.373170] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Expecting reply to msg b74dafb5e05d40a8968a25a7b92349af in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 859.494270] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b74dafb5e05d40a8968a25a7b92349af [ 859.876053] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 859.888855] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 859.889113] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 859.889279] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 859.912755] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 859.912986] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 859.913264] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 859.913643] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 859.913825] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 859.913983] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 859.921389] nova-conductor[54074]: DEBUG nova.quota [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Getting quotas for project 4392957210d14f89af11bf7b1bf7ffc7. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 859.925225] nova-conductor[54074]: DEBUG nova.quota [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Getting quotas for user 85bae9b15b0f43c0bc6e4d4f4c6a28e3 and project 4392957210d14f89af11bf7b1bf7ffc7. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 859.930018] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] [instance: d851f6a6-07aa-4e64-a007-8a42a8ae9c42] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 859.930419] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 859.930624] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 859.930791] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 859.937671] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] [instance: d851f6a6-07aa-4e64-a007-8a42a8ae9c42] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 859.937671] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 859.937671] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 859.937671] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 859.965482] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 859.965745] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 859.965924] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 860.491382] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] [instance: 47b83dbe-d7d8-4875-bb79-95a8fecf4028] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port ae7e19e9-1d82-47c1-96f4-06020432387b, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 47b83dbe-d7d8-4875-bb79-95a8fecf4028 was re-scheduled: Binding failed for port ae7e19e9-1d82-47c1-96f4-06020432387b, please check neutron logs for more information.\n'] [ 860.491948] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 860.492639] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 47b83dbe-d7d8-4875-bb79-95a8fecf4028.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 47b83dbe-d7d8-4875-bb79-95a8fecf4028. [ 860.493078] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] [instance: 47b83dbe-d7d8-4875-bb79-95a8fecf4028] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 47b83dbe-d7d8-4875-bb79-95a8fecf4028. [ 860.516670] nova-conductor[54075]: DEBUG nova.network.neutron [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] [instance: 47b83dbe-d7d8-4875-bb79-95a8fecf4028] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 860.559976] nova-conductor[54075]: DEBUG nova.network.neutron [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] [instance: 47b83dbe-d7d8-4875-bb79-95a8fecf4028] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 860.563133] nova-conductor[54075]: DEBUG nova.network.neutron [None req-9bb53392-410e-4565-9829-8cba0afd3a53 tempest-ServerActionsTestOtherA-442155103 tempest-ServerActionsTestOtherA-442155103-project-member] [instance: 47b83dbe-d7d8-4875-bb79-95a8fecf4028] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 862.296151] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: aa1afca5-8194-4a9d-bcd0-e3e91c15338c] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port f608a2e6-985c-4133-bb07-9b62be09d8d9, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance aa1afca5-8194-4a9d-bcd0-e3e91c15338c was re-scheduled: Binding failed for port f608a2e6-985c-4133-bb07-9b62be09d8d9, please check neutron logs for more information.\n'] [ 862.296151] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 862.296151] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance aa1afca5-8194-4a9d-bcd0-e3e91c15338c.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance aa1afca5-8194-4a9d-bcd0-e3e91c15338c. [ 862.296151] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: aa1afca5-8194-4a9d-bcd0-e3e91c15338c] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance aa1afca5-8194-4a9d-bcd0-e3e91c15338c. [ 862.326146] nova-conductor[54075]: DEBUG nova.network.neutron [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: aa1afca5-8194-4a9d-bcd0-e3e91c15338c] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 862.344947] nova-conductor[54075]: DEBUG nova.network.neutron [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: aa1afca5-8194-4a9d-bcd0-e3e91c15338c] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 862.348654] nova-conductor[54075]: DEBUG nova.network.neutron [None req-d7571335-9041-4af4-97e0-a0db7879f77e tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: aa1afca5-8194-4a9d-bcd0-e3e91c15338c] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 863.356534] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Expecting reply to msg 525f5d7952ed476798b54ca648658b3e in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 863.513255] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 525f5d7952ed476798b54ca648658b3e [ 863.860131] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 863.873578] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 863.873808] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 863.873978] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 863.900099] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 863.900335] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 863.900497] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 863.900863] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 863.901045] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 863.901203] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 863.911579] nova-conductor[54075]: DEBUG nova.quota [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Getting quotas for project 61a866168186462d9d849072a1ff25f1. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 863.914250] nova-conductor[54075]: DEBUG nova.quota [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Getting quotas for user 6349e1aff7d945a6a471b1f4e826b23f and project 61a866168186462d9d849072a1ff25f1. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 863.919051] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 55da10ab-e116-4ead-90ff-c82fffb2dcc6] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 863.919467] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 863.919663] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 863.919824] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 863.924260] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 55da10ab-e116-4ead-90ff-c82fffb2dcc6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 863.925366] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 863.925950] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 863.925950] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 863.941319] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 863.941539] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 863.941702] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 867.556876] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] [instance: 732cf1e3-823d-4769-ad16-f5b492be53d5] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port caac7c6a-04c0-41de-8a4c-52c979260348, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 732cf1e3-823d-4769-ad16-f5b492be53d5 was re-scheduled: Binding failed for port caac7c6a-04c0-41de-8a4c-52c979260348, please check neutron logs for more information.\n'] [ 867.557518] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 867.557771] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 732cf1e3-823d-4769-ad16-f5b492be53d5.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 732cf1e3-823d-4769-ad16-f5b492be53d5. [ 867.558163] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] [instance: 732cf1e3-823d-4769-ad16-f5b492be53d5] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 732cf1e3-823d-4769-ad16-f5b492be53d5. [ 867.586586] nova-conductor[54074]: DEBUG nova.network.neutron [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] [instance: 732cf1e3-823d-4769-ad16-f5b492be53d5] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 867.714826] nova-conductor[54074]: DEBUG nova.network.neutron [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] [instance: 732cf1e3-823d-4769-ad16-f5b492be53d5] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 867.718879] nova-conductor[54074]: DEBUG nova.network.neutron [None req-6c419abe-124e-4c6d-b61e-1f19380fe58a tempest-ServerRescueTestJSONUnderV235-376371306 tempest-ServerRescueTestJSONUnderV235-376371306-project-member] [instance: 732cf1e3-823d-4769-ad16-f5b492be53d5] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 869.918630] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 869.942520] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=3,nova_cell1:INSERT=7 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 869.949124] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=6 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 869.964961] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=6,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 871.209331] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] [instance: 436788b9-92bb-4088-9c24-c2e9a073c09d] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port e91962f5-3d5c-4d08-af11-4df2df76c337, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 436788b9-92bb-4088-9c24-c2e9a073c09d was re-scheduled: Binding failed for port e91962f5-3d5c-4d08-af11-4df2df76c337, please check neutron logs for more information.\n'] [ 871.209916] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 871.210229] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 436788b9-92bb-4088-9c24-c2e9a073c09d.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 436788b9-92bb-4088-9c24-c2e9a073c09d. [ 871.210334] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] [instance: 436788b9-92bb-4088-9c24-c2e9a073c09d] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 436788b9-92bb-4088-9c24-c2e9a073c09d. [ 871.229212] nova-conductor[54074]: DEBUG nova.network.neutron [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] [instance: 436788b9-92bb-4088-9c24-c2e9a073c09d] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 871.243485] nova-conductor[54074]: DEBUG nova.network.neutron [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] [instance: 436788b9-92bb-4088-9c24-c2e9a073c09d] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 871.246019] nova-conductor[54074]: DEBUG nova.network.neutron [None req-cf2a0747-5816-4e46-b6db-fa085d393fe4 tempest-ServerMetadataNegativeTestJSON-911262689 tempest-ServerMetadataNegativeTestJSON-911262689-project-member] [instance: 436788b9-92bb-4088-9c24-c2e9a073c09d] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 873.022020] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] [instance: 53d6d89d-04bb-421d-994c-014830491dfa] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port fcae7974-3037-4637-9e78-4b3d7df2d667, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 53d6d89d-04bb-421d-994c-014830491dfa was re-scheduled: Binding failed for port fcae7974-3037-4637-9e78-4b3d7df2d667, please check neutron logs for more information.\n'] [ 873.022636] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 873.022876] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 53d6d89d-04bb-421d-994c-014830491dfa.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 53d6d89d-04bb-421d-994c-014830491dfa. [ 873.023216] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] [instance: 53d6d89d-04bb-421d-994c-014830491dfa] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 53d6d89d-04bb-421d-994c-014830491dfa. [ 873.054303] nova-conductor[54075]: DEBUG nova.network.neutron [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] [instance: 53d6d89d-04bb-421d-994c-014830491dfa] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 873.276334] nova-conductor[54075]: DEBUG nova.network.neutron [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] [instance: 53d6d89d-04bb-421d-994c-014830491dfa] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 873.279755] nova-conductor[54075]: DEBUG nova.network.neutron [None req-1a371740-4d2c-404d-9660-d265d4a3da45 tempest-ServerActionsTestJSON-1289460322 tempest-ServerActionsTestJSON-1289460322-project-member] [instance: 53d6d89d-04bb-421d-994c-014830491dfa] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 873.906509] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 873.927576] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=3,nova_cell1:INSERT=7 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 873.933944] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=5,nova_api:UPDATE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 873.938411] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=7,nova_api:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 874.800465] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 3ada5090-7219-4835-b508-2188501ae5e4] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port e599aa10-a1f1-41b4-933c-2956e0b7e627, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 3ada5090-7219-4835-b508-2188501ae5e4 was re-scheduled: Binding failed for port e599aa10-a1f1-41b4-933c-2956e0b7e627, please check neutron logs for more information.\n'] [ 874.801219] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 874.801519] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3ada5090-7219-4835-b508-2188501ae5e4.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3ada5090-7219-4835-b508-2188501ae5e4. [ 874.801818] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 3ada5090-7219-4835-b508-2188501ae5e4] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3ada5090-7219-4835-b508-2188501ae5e4. [ 874.830110] nova-conductor[54074]: DEBUG nova.network.neutron [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 3ada5090-7219-4835-b508-2188501ae5e4] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 874.853400] nova-conductor[54074]: DEBUG nova.network.neutron [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 3ada5090-7219-4835-b508-2188501ae5e4] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 874.863608] nova-conductor[54074]: DEBUG nova.network.neutron [None req-5dc46801-54e7-4816-8481-80ed741cbbc9 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 3ada5090-7219-4835-b508-2188501ae5e4] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 876.689101] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 6163fcd4-cfe4-4432-ba8d-665319fa11ed] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 4708138f-6f39-4fbc-8800-010c527d3c13, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 6163fcd4-cfe4-4432-ba8d-665319fa11ed was re-scheduled: Binding failed for port 4708138f-6f39-4fbc-8800-010c527d3c13, please check neutron logs for more information.\n'] [ 876.689713] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 876.689943] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6163fcd4-cfe4-4432-ba8d-665319fa11ed.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6163fcd4-cfe4-4432-ba8d-665319fa11ed. [ 876.690158] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 6163fcd4-cfe4-4432-ba8d-665319fa11ed] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6163fcd4-cfe4-4432-ba8d-665319fa11ed. [ 876.712774] nova-conductor[54075]: DEBUG nova.network.neutron [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 6163fcd4-cfe4-4432-ba8d-665319fa11ed] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 876.763493] nova-conductor[54075]: DEBUG nova.network.neutron [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 6163fcd4-cfe4-4432-ba8d-665319fa11ed] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 876.767032] nova-conductor[54075]: DEBUG nova.network.neutron [None req-fc11f9d3-77d4-4557-afb7-a5d4edf5e862 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 6163fcd4-cfe4-4432-ba8d-665319fa11ed] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 877.581039] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Expecting reply to msg 3b5282eb77e444fca4451f94c4f4c5db in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 877.714802] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3b5282eb77e444fca4451f94c4f4c5db [ 878.089775] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 878.112887] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 878.113324] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 878.113688] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 878.163728] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 878.163966] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 878.164164] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 878.164551] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 878.164732] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 878.164888] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 878.174208] nova-conductor[54075]: DEBUG nova.quota [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Getting quotas for project 21cdf8675fb347c2874d912dcb8ac002. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 878.176987] nova-conductor[54075]: DEBUG nova.quota [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Getting quotas for user 23f9946393284244aca0c29201c37fa4 and project 21cdf8675fb347c2874d912dcb8ac002. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 878.183613] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 87dff872-a469-465f-9c74-4524a2eab013] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 878.184112] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 878.185310] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 878.185310] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 878.188027] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 87dff872-a469-465f-9c74-4524a2eab013] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 878.188508] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 878.188866] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 878.189263] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 878.209668] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 878.209913] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 878.210372] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 878.299474] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Expecting reply to msg 58277e3734e2475bad2df93396b662de in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 878.439492] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 58277e3734e2475bad2df93396b662de [ 878.654047] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Expecting reply to msg 80e92b949a194b7386f59abd7b491291 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 878.778845] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 80e92b949a194b7386f59abd7b491291 [ 878.802382] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 878.814938] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 878.815158] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 878.815318] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 878.846380] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 878.846610] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 878.846779] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 878.847145] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 878.847328] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 878.847482] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 878.854889] nova-conductor[54075]: DEBUG nova.quota [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Getting quotas for project aa59611b27bb41beb282e68ccfa6fadc. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 878.856856] nova-conductor[54075]: DEBUG nova.quota [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Getting quotas for user ba5969be1a254281b4dffd81aa84ae7e and project aa59611b27bb41beb282e68ccfa6fadc. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 878.862103] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: bab79bb6-1638-4eee-812d-da1372134873] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 878.862723] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 878.863037] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 878.863302] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 878.868339] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: bab79bb6-1638-4eee-812d-da1372134873] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 878.869155] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 878.869521] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 878.869800] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 878.887406] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 878.887806] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 878.888117] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 879.156432] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 879.171314] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 879.171656] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 879.171929] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 879.198030] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 879.198260] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 879.198424] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 879.198781] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 879.198979] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 879.199102] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 879.206887] nova-conductor[54074]: DEBUG nova.quota [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Getting quotas for project e35f4d42ead34229b32278931e92dc0c. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 879.209969] nova-conductor[54074]: DEBUG nova.quota [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Getting quotas for user 558eac5e9e0e46499aed7185a9b33af7 and project e35f4d42ead34229b32278931e92dc0c. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 879.216460] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] [instance: 5f58014c-e132-4fad-9ba7-bc183318200f] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 879.216893] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 879.217096] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 879.217300] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 879.219784] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] [instance: 5f58014c-e132-4fad-9ba7-bc183318200f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 879.220522] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 879.220761] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 879.220934] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 879.234653] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 879.234869] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 879.235044] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 880.716797] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] [instance: 252b7e84-4f91-4078-a81c-392d622b6ce2] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 906773bb-0ccc-45dc-9288-5601845118c0, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 252b7e84-4f91-4078-a81c-392d622b6ce2 was re-scheduled: Binding failed for port 906773bb-0ccc-45dc-9288-5601845118c0, please check neutron logs for more information.\n'] [ 880.716797] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 880.716797] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 252b7e84-4f91-4078-a81c-392d622b6ce2.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 252b7e84-4f91-4078-a81c-392d622b6ce2. [ 880.717219] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] [instance: 252b7e84-4f91-4078-a81c-392d622b6ce2] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 252b7e84-4f91-4078-a81c-392d622b6ce2. [ 880.739845] nova-conductor[54074]: DEBUG nova.network.neutron [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] [instance: 252b7e84-4f91-4078-a81c-392d622b6ce2] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 880.759508] nova-conductor[54074]: DEBUG nova.network.neutron [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] [instance: 252b7e84-4f91-4078-a81c-392d622b6ce2] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 880.764638] nova-conductor[54074]: DEBUG nova.network.neutron [None req-f22245ba-14b0-41a1-a65f-5dbae505921b tempest-ServerMetadataTestJSON-985696139 tempest-ServerMetadataTestJSON-985696139-project-member] [instance: 252b7e84-4f91-4078-a81c-392d622b6ce2] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 883.701597] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Expecting reply to msg 414815b5c3914a0cbc9022791b030139 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 883.828289] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 414815b5c3914a0cbc9022791b030139 [ 884.204100] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 884.216576] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 884.216793] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 884.216985] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 884.247430] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 884.247639] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 884.247824] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 884.248282] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 884.248463] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 884.248620] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 884.256321] nova-conductor[54074]: DEBUG nova.quota [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Getting quotas for project 393d372817d746aa82df4745529f66f3. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 884.258495] nova-conductor[54074]: DEBUG nova.quota [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Getting quotas for user 0537f3cad9bb40dc9e7a570192e905fb and project 393d372817d746aa82df4745529f66f3. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 884.263512] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] [instance: 0be23f0b-0c43-4eb1-8572-b8ef84f6c5b3] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 884.263921] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 884.264133] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 884.264300] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 884.269517] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] [instance: 0be23f0b-0c43-4eb1-8572-b8ef84f6c5b3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 884.270170] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 884.270390] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 884.270583] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 884.290772] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 884.290980] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 884.291147] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-5213cfec-48e3-4e2b-9180-e27103bafb9a tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 884.351797] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=411,nova_cell1:UPDATE=217,nova_cell1:INSERT=31,nova_cell1:SAVEPOINT=41,nova_cell1:RELEASE=41,nova_cell1:DELETE=2 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 887.037703] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 3c0ed5ff-4232-4ab5-a91c-d82da0008f8e] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 364d95c6-b690-4798-ac6c-92324d6dbd83, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 3c0ed5ff-4232-4ab5-a91c-d82da0008f8e was re-scheduled: Binding failed for port 364d95c6-b690-4798-ac6c-92324d6dbd83, please check neutron logs for more information.\n'] [ 887.038351] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 887.038608] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3c0ed5ff-4232-4ab5-a91c-d82da0008f8e.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3c0ed5ff-4232-4ab5-a91c-d82da0008f8e. [ 887.038984] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 3c0ed5ff-4232-4ab5-a91c-d82da0008f8e] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3c0ed5ff-4232-4ab5-a91c-d82da0008f8e. [ 887.060805] nova-conductor[54075]: DEBUG nova.network.neutron [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 3c0ed5ff-4232-4ab5-a91c-d82da0008f8e] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 887.084668] nova-conductor[54075]: DEBUG nova.network.neutron [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 3c0ed5ff-4232-4ab5-a91c-d82da0008f8e] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 887.088116] nova-conductor[54075]: DEBUG nova.network.neutron [None req-77b948e5-349d-436a-9f94-c4807bcef6cc tempest-ImagesTestJSON-1803525400 tempest-ImagesTestJSON-1803525400-project-member] [instance: 3c0ed5ff-4232-4ab5-a91c-d82da0008f8e] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 888.402148] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=403,nova_cell1:UPDATE=197,nova_cell1:INSERT=30,nova_cell1:SAVEPOINT=40,nova_cell1:RELEASE=40 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 888.852206] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=2 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 888.871939] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=6,nova_cell1:INSERT=14 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 888.882183] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=11,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 888.885654] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=13,nova_api:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 889.345696] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d84d21fb-8570-4504-95b8-f44dca49ad2e tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Targeting cell 89980bcd-ea65-4338-957c-d9d250569837(cell1) for conductor method rebuild_instance {{(pid=54075) wrapper /opt/stack/nova/nova/conductor/manager.py:95}} [ 889.346573] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d84d21fb-8570-4504-95b8-f44dca49ad2e tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 889.352421] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d84d21fb-8570-4504-95b8-f44dca49ad2e tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 889.352868] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-d84d21fb-8570-4504-95b8-f44dca49ad2e tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 889.371186] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d84d21fb-8570-4504-95b8-f44dca49ad2e tempest-ServersAdmin275Test-1979464108 tempest-ServersAdmin275Test-1979464108-project-member] [instance: 309a7bae-82f5-4b9e-ac86-e0f1803f2585] No migration record for the rebuild/evacuate request. {{(pid=54075) rebuild_instance /opt/stack/nova/nova/conductor/manager.py:1227}} [ 892.688640] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Expecting reply to msg 11ae3eac214d4c1dbce94d05634c8f61 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 892.839660] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 11ae3eac214d4c1dbce94d05634c8f61 [ 893.192858] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 893.207528] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 893.207748] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 893.207909] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 893.278358] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 893.278576] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 893.278741] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 893.279191] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 893.279294] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 893.279456] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 893.288101] nova-conductor[54074]: DEBUG nova.quota [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Getting quotas for project 7c386f1cfdf24c9ab1ab01f5a00ed743. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 893.291415] nova-conductor[54074]: DEBUG nova.quota [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Getting quotas for user de2402e819b745498f3b594ac0c25d69 and project 7c386f1cfdf24c9ab1ab01f5a00ed743. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 893.296339] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] [instance: 8bd1a8aa-844b-47ca-9296-0c30af695984] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 893.296818] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 893.297019] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 893.297185] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 893.308445] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] [instance: 8bd1a8aa-844b-47ca-9296-0c30af695984] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 893.309426] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 893.309648] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 893.309812] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 893.338001] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 893.338286] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 893.338497] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 898.439186] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-4d7c83a5-f13b-4e04-aab7-5644b6fd69f5 tempest-ServersAdmin275Test-2021021546 tempest-ServersAdmin275Test-2021021546-project-admin] Targeting cell 89980bcd-ea65-4338-957c-d9d250569837(cell1) for conductor method rebuild_instance {{(pid=54074) wrapper /opt/stack/nova/nova/conductor/manager.py:95}} [ 898.439186] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4d7c83a5-f13b-4e04-aab7-5644b6fd69f5 tempest-ServersAdmin275Test-2021021546 tempest-ServersAdmin275Test-2021021546-project-admin] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 898.439186] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4d7c83a5-f13b-4e04-aab7-5644b6fd69f5 tempest-ServersAdmin275Test-2021021546 tempest-ServersAdmin275Test-2021021546-project-admin] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 898.439186] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-4d7c83a5-f13b-4e04-aab7-5644b6fd69f5 tempest-ServersAdmin275Test-2021021546 tempest-ServersAdmin275Test-2021021546-project-admin] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 898.447108] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-4d7c83a5-f13b-4e04-aab7-5644b6fd69f5 tempest-ServersAdmin275Test-2021021546 tempest-ServersAdmin275Test-2021021546-project-admin] [instance: 309a7bae-82f5-4b9e-ac86-e0f1803f2585] No migration record for the rebuild/evacuate request. {{(pid=54074) rebuild_instance /opt/stack/nova/nova/conductor/manager.py:1227}} [ 899.377914] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=2 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 899.405119] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=4,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 903.282682] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=3 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 904.521043] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] [instance: f453b695-8abd-44fa-8468-75c6aaeec19a] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b79d0813-93ae-42da-a060-e3a3b0e18d63, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance f453b695-8abd-44fa-8468-75c6aaeec19a was re-scheduled: Binding failed for port b79d0813-93ae-42da-a060-e3a3b0e18d63, please check neutron logs for more information.\n'] [ 904.521561] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 904.521788] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f453b695-8abd-44fa-8468-75c6aaeec19a.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f453b695-8abd-44fa-8468-75c6aaeec19a. [ 904.522096] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] [instance: f453b695-8abd-44fa-8468-75c6aaeec19a] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f453b695-8abd-44fa-8468-75c6aaeec19a. [ 904.542795] nova-conductor[54074]: DEBUG nova.network.neutron [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] [instance: f453b695-8abd-44fa-8468-75c6aaeec19a] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 904.557639] nova-conductor[54074]: DEBUG nova.network.neutron [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] [instance: f453b695-8abd-44fa-8468-75c6aaeec19a] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 904.560532] nova-conductor[54074]: DEBUG nova.network.neutron [None req-8da93c1e-ac61-4029-a339-d79994345f76 tempest-ServersTestBootFromVolume-235730665 tempest-ServersTestBootFromVolume-235730665-project-member] [instance: f453b695-8abd-44fa-8468-75c6aaeec19a] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 906.329678] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: e2e09174-6ba1-44ad-ba3e-cdcae5a2d698] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 492e9847-c7bd-424b-b9ba-eed84aef6eb0, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance e2e09174-6ba1-44ad-ba3e-cdcae5a2d698 was re-scheduled: Binding failed for port 492e9847-c7bd-424b-b9ba-eed84aef6eb0, please check neutron logs for more information.\n'] [ 906.330223] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 906.330445] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e2e09174-6ba1-44ad-ba3e-cdcae5a2d698.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e2e09174-6ba1-44ad-ba3e-cdcae5a2d698. [ 906.330644] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: e2e09174-6ba1-44ad-ba3e-cdcae5a2d698] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e2e09174-6ba1-44ad-ba3e-cdcae5a2d698. [ 906.357775] nova-conductor[54074]: DEBUG nova.network.neutron [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: e2e09174-6ba1-44ad-ba3e-cdcae5a2d698] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 906.378727] nova-conductor[54074]: DEBUG nova.network.neutron [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: e2e09174-6ba1-44ad-ba3e-cdcae5a2d698] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 906.381966] nova-conductor[54074]: DEBUG nova.network.neutron [None req-97e001d4-24dc-41bf-acc1-538381c890c3 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: e2e09174-6ba1-44ad-ba3e-cdcae5a2d698] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 907.399297] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-84eb6a41-eb87-4d12-9b36-fcbac6752818 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Expecting reply to msg 8d38d1e4c4914987a22666d32a90a5cb in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 907.559997] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8d38d1e4c4914987a22666d32a90a5cb [ 907.901862] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-84eb6a41-eb87-4d12-9b36-fcbac6752818 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 907.908364] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-84eb6a41-eb87-4d12-9b36-fcbac6752818 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 5a47dcf1-57f8-4be5-8ef2-f88f96d39c5e] While scheduling instance, the build request was already deleted. {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1706}} [ 907.935866] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-84eb6a41-eb87-4d12-9b36-fcbac6752818 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 907.936160] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-84eb6a41-eb87-4d12-9b36-fcbac6752818 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 907.936340] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-84eb6a41-eb87-4d12-9b36-fcbac6752818 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 907.936707] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-84eb6a41-eb87-4d12-9b36-fcbac6752818 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 907.936888] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-84eb6a41-eb87-4d12-9b36-fcbac6752818 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 907.937047] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-84eb6a41-eb87-4d12-9b36-fcbac6752818 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 907.945384] nova-conductor[54074]: DEBUG nova.quota [None req-84eb6a41-eb87-4d12-9b36-fcbac6752818 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Getting quotas for project 71a669cc62964b05bdaa71442c08a566. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 907.947536] nova-conductor[54074]: DEBUG nova.quota [None req-84eb6a41-eb87-4d12-9b36-fcbac6752818 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Getting quotas for user 2d003a5943d141cc9165e43148dec381 and project 71a669cc62964b05bdaa71442c08a566. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 907.957087] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Expecting reply to msg 8c8301a77a95429bafa9fe35cd5ef805 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 908.080488] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8c8301a77a95429bafa9fe35cd5ef805 [ 908.465869] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 908.479703] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 908.479924] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 908.480096] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 908.517852] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 908.518318] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 908.518615] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 908.519101] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 908.519455] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 908.519707] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 908.532030] nova-conductor[54075]: DEBUG nova.quota [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Getting quotas for project 71a669cc62964b05bdaa71442c08a566. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 908.535150] nova-conductor[54075]: DEBUG nova.quota [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Getting quotas for user 2d003a5943d141cc9165e43148dec381 and project 71a669cc62964b05bdaa71442c08a566. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 908.544700] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 27fed863-1e27-4258-8b43-b8cd23e3c1c0] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 908.545418] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 908.545736] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 908.546028] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 908.551252] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 27fed863-1e27-4258-8b43-b8cd23e3c1c0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 908.552106] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 908.552427] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 908.552695] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 908.581597] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 908.581965] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 908.582266] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 910.523844] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Expecting reply to msg cbda1b266def49e6ad808417075bb1af in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 910.647133] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg cbda1b266def49e6ad808417075bb1af [ 911.027161] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 911.041139] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 911.041460] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 911.041779] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 911.074651] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 911.075011] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 911.075251] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 911.075698] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 911.075988] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 911.076191] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 911.084159] nova-conductor[54075]: DEBUG nova.quota [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Getting quotas for project 7e2b950a9c8f42ffbe0d4ced7a3f289c. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 911.086333] nova-conductor[54075]: DEBUG nova.quota [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Getting quotas for user 28c774de080644e8979aad3cdb0c4400 and project 7e2b950a9c8f42ffbe0d4ced7a3f289c. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 911.091655] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] [instance: 228d2a6d-6c16-472c-9326-2e4576d9648c] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 911.092240] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 911.092532] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 911.092862] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 911.097992] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] [instance: 228d2a6d-6c16-472c-9326-2e4576d9648c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 911.098195] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 911.098866] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 911.099395] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 911.113478] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 911.113715] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 911.113867] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 912.744366] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] [instance: 3f99ec88-f05f-4583-b08b-d40fb37e275e] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 2b78f53d-d85a-41f3-91bb-f8df87915fcc, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 3f99ec88-f05f-4583-b08b-d40fb37e275e was re-scheduled: Binding failed for port 2b78f53d-d85a-41f3-91bb-f8df87915fcc, please check neutron logs for more information.\n'] [ 912.744931] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 912.745174] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3f99ec88-f05f-4583-b08b-d40fb37e275e.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3f99ec88-f05f-4583-b08b-d40fb37e275e. [ 912.745374] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] [instance: 3f99ec88-f05f-4583-b08b-d40fb37e275e] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3f99ec88-f05f-4583-b08b-d40fb37e275e. [ 912.766227] nova-conductor[54074]: DEBUG nova.network.neutron [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] [instance: 3f99ec88-f05f-4583-b08b-d40fb37e275e] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 912.780324] nova-conductor[54074]: DEBUG nova.network.neutron [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] [instance: 3f99ec88-f05f-4583-b08b-d40fb37e275e] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 912.783652] nova-conductor[54074]: DEBUG nova.network.neutron [None req-18395fe6-1457-4f1a-ae2e-d1b66ac44bfb tempest-AttachVolumeTestJSON-1360449787 tempest-AttachVolumeTestJSON-1360449787-project-member] [instance: 3f99ec88-f05f-4583-b08b-d40fb37e275e] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 914.607496] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 1a38b70f-eabe-4b11-a371-cf971184211f] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 2fbe7b17-bbb3-4849-8e6c-59f3ea6695f9, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 1a38b70f-eabe-4b11-a371-cf971184211f was re-scheduled: Binding failed for port 2fbe7b17-bbb3-4849-8e6c-59f3ea6695f9, please check neutron logs for more information.\n'] [ 914.608171] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 914.608505] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 1a38b70f-eabe-4b11-a371-cf971184211f.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 1a38b70f-eabe-4b11-a371-cf971184211f. [ 914.608912] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 1a38b70f-eabe-4b11-a371-cf971184211f] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 1a38b70f-eabe-4b11-a371-cf971184211f. [ 914.642038] nova-conductor[54074]: DEBUG nova.network.neutron [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 1a38b70f-eabe-4b11-a371-cf971184211f] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 914.656084] nova-conductor[54074]: DEBUG nova.network.neutron [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 1a38b70f-eabe-4b11-a371-cf971184211f] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 914.659135] nova-conductor[54074]: DEBUG nova.network.neutron [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 1a38b70f-eabe-4b11-a371-cf971184211f] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 916.323693] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 8a6d10c8-bd2b-40dd-9897-8f30223abe81] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 84f69f0f-65c8-4a94-811d-05ed8f7a266a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 8a6d10c8-bd2b-40dd-9897-8f30223abe81 was re-scheduled: Binding failed for port 84f69f0f-65c8-4a94-811d-05ed8f7a266a, please check neutron logs for more information.\n'] [ 916.324333] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 916.324557] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8a6d10c8-bd2b-40dd-9897-8f30223abe81.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8a6d10c8-bd2b-40dd-9897-8f30223abe81. [ 916.324917] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 8a6d10c8-bd2b-40dd-9897-8f30223abe81] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8a6d10c8-bd2b-40dd-9897-8f30223abe81. [ 916.354251] nova-conductor[54075]: DEBUG nova.network.neutron [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 8a6d10c8-bd2b-40dd-9897-8f30223abe81] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 916.369709] nova-conductor[54075]: DEBUG nova.network.neutron [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 8a6d10c8-bd2b-40dd-9897-8f30223abe81] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 916.375951] nova-conductor[54075]: DEBUG nova.network.neutron [None req-d31f8600-9054-4158-b0cc-126d1d749c80 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 8a6d10c8-bd2b-40dd-9897-8f30223abe81] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 917.942304] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Expecting reply to msg dacfa84cf748448abba6a5d340c2fca0 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 917.943482] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=14,nova_cell1:SAVEPOINT=3,nova_cell1:INSERT=25,nova_cell1:RELEASE=3,nova_cell1:UPDATE=3 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 917.944059] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 917.956102] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=22,nova_api:UPDATE=1,nova_api:DELETE=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 918.148920] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg dacfa84cf748448abba6a5d340c2fca0 [ 918.446196] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Took 0.51 seconds to select destinations for 2 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 918.463491] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 918.463491] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 918.463491] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 918.506613] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 918.506859] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 918.507012] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 918.531453] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 918.531678] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 918.532211] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 918.532725] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 918.532944] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 918.533112] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 918.543822] nova-conductor[54074]: DEBUG nova.quota [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Getting quotas for project 4e10657828d9480b948e59b98490572b. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 918.548360] nova-conductor[54074]: DEBUG nova.quota [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Getting quotas for user feae790f4343445c86cfb1b39cb9636e and project 4e10657828d9480b948e59b98490572b. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 918.553328] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: a879b81d-fb5a-483b-9c2a-4a5c416c1caa] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 918.553762] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 918.553960] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 918.554123] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 918.559779] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: a879b81d-fb5a-483b-9c2a-4a5c416c1caa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 918.565388] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 918.567188] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 918.567188] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 918.606778] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 918.607020] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 918.607180] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 918.613228] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 9ab3a71f-7e26-4d29-b006-6dbebcee16e1] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 918.614561] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 918.614561] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 918.614561] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 918.618374] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 9ab3a71f-7e26-4d29-b006-6dbebcee16e1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 918.618983] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 918.619251] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 918.619425] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 918.637327] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 918.637596] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 918.637803] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 919.073820] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Expecting reply to msg 4e4f5b80149343e7804bd89dafc0e182 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 919.104992] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] [instance: a276656a-67b0-4ceb-918f-cfb323ed09fd] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port e979a70d-ad81-4d49-a987-fcf30691d88c, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance a276656a-67b0-4ceb-918f-cfb323ed09fd was re-scheduled: Binding failed for port e979a70d-ad81-4d49-a987-fcf30691d88c, please check neutron logs for more information.\n'] [ 919.105671] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 919.105905] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a276656a-67b0-4ceb-918f-cfb323ed09fd.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a276656a-67b0-4ceb-918f-cfb323ed09fd. [ 919.106484] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] [instance: a276656a-67b0-4ceb-918f-cfb323ed09fd] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a276656a-67b0-4ceb-918f-cfb323ed09fd. [ 919.134667] nova-conductor[54074]: DEBUG nova.network.neutron [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] [instance: a276656a-67b0-4ceb-918f-cfb323ed09fd] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 919.153409] nova-conductor[54074]: DEBUG nova.network.neutron [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] [instance: a276656a-67b0-4ceb-918f-cfb323ed09fd] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 919.156559] nova-conductor[54074]: DEBUG nova.network.neutron [None req-33f21230-5d02-4b63-b679-8725564d012a tempest-ServersNegativeTestMultiTenantJSON-803762167 tempest-ServersNegativeTestMultiTenantJSON-803762167-project-member] [instance: a276656a-67b0-4ceb-918f-cfb323ed09fd] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 919.201094] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4e4f5b80149343e7804bd89dafc0e182 [ 919.578758] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 919.593295] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 919.594844] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 919.595117] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 919.631807] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 919.632143] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 919.632389] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 919.632825] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 919.633075] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 919.633303] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 919.642967] nova-conductor[54075]: DEBUG nova.quota [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Getting quotas for project 98934316e57a4ea69b2bb5a2f2aaf251. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 919.646206] nova-conductor[54075]: DEBUG nova.quota [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Getting quotas for user 2883d8b8ab764050a13c8b3a56318c34 and project 98934316e57a4ea69b2bb5a2f2aaf251. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 919.651817] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: b04cc451-a497-474f-90dd-282a469ff3c2] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 919.652534] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 919.653047] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 919.653251] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 919.656382] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: b04cc451-a497-474f-90dd-282a469ff3c2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 919.657256] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 919.657685] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 919.657934] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 919.677031] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 919.677389] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 919.677623] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 920.848118] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] [instance: d851f6a6-07aa-4e64-a007-8a42a8ae9c42] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port cd11637a-e069-4299-9bbb-23a2b33fa21c, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance d851f6a6-07aa-4e64-a007-8a42a8ae9c42 was re-scheduled: Binding failed for port cd11637a-e069-4299-9bbb-23a2b33fa21c, please check neutron logs for more information.\n'] [ 920.848769] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 920.849089] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d851f6a6-07aa-4e64-a007-8a42a8ae9c42.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d851f6a6-07aa-4e64-a007-8a42a8ae9c42. [ 920.855977] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] [instance: d851f6a6-07aa-4e64-a007-8a42a8ae9c42] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d851f6a6-07aa-4e64-a007-8a42a8ae9c42. [ 920.879718] nova-conductor[54075]: DEBUG nova.network.neutron [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] [instance: d851f6a6-07aa-4e64-a007-8a42a8ae9c42] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 920.905839] nova-conductor[54075]: DEBUG nova.network.neutron [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] [instance: d851f6a6-07aa-4e64-a007-8a42a8ae9c42] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 920.909855] nova-conductor[54075]: DEBUG nova.network.neutron [None req-08ae84bb-a61c-483a-822d-5b6111e21965 tempest-ServersTestMultiNic-293111283 tempest-ServersTestMultiNic-293111283-project-member] [instance: d851f6a6-07aa-4e64-a007-8a42a8ae9c42] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 928.540030] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 928.622587] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=5,nova_cell1:INSERT=14 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 928.635715] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=33,nova_api:UPDATE=4,nova_api:DELETE=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 928.636514] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=8,nova_api:DELETE=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 929.438076] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 55da10ab-e116-4ead-90ff-c82fffb2dcc6] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 792b62d9-f0ba-485b-9131-66603f84de93, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 55da10ab-e116-4ead-90ff-c82fffb2dcc6 was re-scheduled: Binding failed for port 792b62d9-f0ba-485b-9131-66603f84de93, please check neutron logs for more information.\n'] [ 929.438651] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 929.438876] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 55da10ab-e116-4ead-90ff-c82fffb2dcc6.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 55da10ab-e116-4ead-90ff-c82fffb2dcc6. [ 929.439088] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 55da10ab-e116-4ead-90ff-c82fffb2dcc6] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 55da10ab-e116-4ead-90ff-c82fffb2dcc6. [ 929.462138] nova-conductor[54074]: DEBUG nova.network.neutron [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 55da10ab-e116-4ead-90ff-c82fffb2dcc6] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 929.479417] nova-conductor[54074]: DEBUG nova.network.neutron [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 55da10ab-e116-4ead-90ff-c82fffb2dcc6] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 929.482763] nova-conductor[54074]: DEBUG nova.network.neutron [None req-6e797b24-cf71-4b5c-9170-8bd5c31743d2 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: 55da10ab-e116-4ead-90ff-c82fffb2dcc6] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 929.637844] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=3 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 929.660265] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=23,nova_cell1:SAVEPOINT=2,nova_cell1:RELEASE=2 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 929.670141] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=17,nova_api:DELETE=2,nova_api:UPDATE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 929.672125] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=19,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 930.837458] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Expecting reply to msg f4268b712f4f49549aea6f7a9558781f in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 930.983627] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f4268b712f4f49549aea6f7a9558781f [ 931.340159] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 931.354765] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 931.355047] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 931.355234] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 931.386032] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 931.386399] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 931.386659] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 931.387062] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 931.387258] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 931.387420] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 931.396558] nova-conductor[54074]: DEBUG nova.quota [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Getting quotas for project 61a866168186462d9d849072a1ff25f1. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 931.400186] nova-conductor[54074]: DEBUG nova.quota [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Getting quotas for user 6349e1aff7d945a6a471b1f4e826b23f and project 61a866168186462d9d849072a1ff25f1. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 931.405232] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: d622ca94-7f5c-47f4-8077-ff37f64eea02] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 931.405690] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 931.406094] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 931.406094] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 931.409460] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: d622ca94-7f5c-47f4-8077-ff37f64eea02] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 931.410102] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 931.410309] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 931.410501] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 931.426099] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 931.427024] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 931.427335] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 936.742747] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 87dff872-a469-465f-9c74-4524a2eab013] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port c42dbb9c-5fa1-493e-8235-912a6dd1e291, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 87dff872-a469-465f-9c74-4524a2eab013 was re-scheduled: Binding failed for port c42dbb9c-5fa1-493e-8235-912a6dd1e291, please check neutron logs for more information.\n'] [ 936.742747] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 936.742747] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 87dff872-a469-465f-9c74-4524a2eab013.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 87dff872-a469-465f-9c74-4524a2eab013. [ 936.742747] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 87dff872-a469-465f-9c74-4524a2eab013] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 87dff872-a469-465f-9c74-4524a2eab013. [ 936.764302] nova-conductor[54074]: DEBUG nova.network.neutron [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 87dff872-a469-465f-9c74-4524a2eab013] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 936.780391] nova-conductor[54074]: DEBUG nova.network.neutron [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 87dff872-a469-465f-9c74-4524a2eab013] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 936.783692] nova-conductor[54074]: DEBUG nova.network.neutron [None req-4a9a2bdf-5492-4120-8c29-b5c2133cce17 tempest-AttachInterfacesTestJSON-1305547194 tempest-AttachInterfacesTestJSON-1305547194-project-member] [instance: 87dff872-a469-465f-9c74-4524a2eab013] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 938.453463] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: bab79bb6-1638-4eee-812d-da1372134873] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 711f1545-3683-4e94-8275-4892d964343b, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance bab79bb6-1638-4eee-812d-da1372134873 was re-scheduled: Binding failed for port 711f1545-3683-4e94-8275-4892d964343b, please check neutron logs for more information.\n'] [ 938.454040] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 938.454268] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance bab79bb6-1638-4eee-812d-da1372134873.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance bab79bb6-1638-4eee-812d-da1372134873. [ 938.454852] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: bab79bb6-1638-4eee-812d-da1372134873] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance bab79bb6-1638-4eee-812d-da1372134873. [ 938.498601] nova-conductor[54074]: DEBUG nova.network.neutron [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: bab79bb6-1638-4eee-812d-da1372134873] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 938.548951] nova-conductor[54074]: DEBUG nova.network.neutron [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: bab79bb6-1638-4eee-812d-da1372134873] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 938.551745] nova-conductor[54074]: DEBUG nova.network.neutron [None req-d9f75258-1b52-46c2-9683-603460492b4b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: bab79bb6-1638-4eee-812d-da1372134873] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 939.814101] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Expecting reply to msg ae39c4e1b81342d8bcd556fbf1b34cee in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 939.962838] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ae39c4e1b81342d8bcd556fbf1b34cee [ 940.090053] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-04340e57-c9b7-47e9-83d9-a9bcc609b11d tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Targeting cell 89980bcd-ea65-4338-957c-d9d250569837(cell1) for conductor method rebuild_instance {{(pid=54075) wrapper /opt/stack/nova/nova/conductor/manager.py:95}} [ 940.090279] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-04340e57-c9b7-47e9-83d9-a9bcc609b11d tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 940.090478] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-04340e57-c9b7-47e9-83d9-a9bcc609b11d tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 940.090639] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-04340e57-c9b7-47e9-83d9-a9bcc609b11d tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 940.099037] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-04340e57-c9b7-47e9-83d9-a9bcc609b11d tempest-ServerShowV254Test-2031808440 tempest-ServerShowV254Test-2031808440-project-member] [instance: 0be23f0b-0c43-4eb1-8572-b8ef84f6c5b3] No migration record for the rebuild/evacuate request. {{(pid=54075) rebuild_instance /opt/stack/nova/nova/conductor/manager.py:1227}} [ 940.317892] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 940.331989] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 940.332300] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 940.332748] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 940.353809] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Expecting reply to msg 0145b19a9052456ebdd8a30a7fe4cf06 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 940.376951] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 940.377526] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 940.378115] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 940.378517] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 940.378709] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 940.378869] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 940.390874] nova-conductor[54075]: DEBUG nova.quota [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Getting quotas for project 760ebbb02a9c44abb44ea37f93694ada. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 940.393755] nova-conductor[54075]: DEBUG nova.quota [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Getting quotas for user 24290d4ecad54e0083310fcd3c199cb8 and project 760ebbb02a9c44abb44ea37f93694ada. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 940.398541] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] [instance: 49a0249a-f322-47f6-b723-2af2b701902c] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 940.401159] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 940.401375] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 940.401541] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 940.411657] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] [instance: 49a0249a-f322-47f6-b723-2af2b701902c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 940.412256] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 940.412463] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 940.412626] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 940.438341] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 940.438599] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 940.438727] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 940.505120] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0145b19a9052456ebdd8a30a7fe4cf06 [ 940.856909] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 940.867489] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 940.867727] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 940.867904] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 940.909573] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 940.909811] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 940.909977] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 940.912131] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 940.912414] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 940.912602] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 940.923550] nova-conductor[54075]: DEBUG nova.quota [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Getting quotas for project aa59611b27bb41beb282e68ccfa6fadc. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 940.928526] nova-conductor[54075]: DEBUG nova.quota [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Getting quotas for user ba5969be1a254281b4dffd81aa84ae7e and project aa59611b27bb41beb282e68ccfa6fadc. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 940.934322] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: f9cb0f9d-fcc7-4241-b6db-ce1cf8f3ed54] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 940.935135] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 940.935425] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 940.935658] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 940.951050] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: f9cb0f9d-fcc7-4241-b6db-ce1cf8f3ed54] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 940.951893] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 940.952235] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 940.952482] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 940.980202] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 940.980202] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 940.980202] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 941.392026] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 941.412821] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=3,nova_cell1:INSERT=7 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 941.421019] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=7,nova_api:UPDATE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 941.423740] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=5,nova_api:DELETE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 944.414164] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=373,nova_cell1:SAVEPOINT=38,nova_cell1:RELEASE=38,nova_cell1:UPDATE=195,nova_cell1:INSERT=19,nova_cell1:DELETE=4 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 947.055150] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Expecting reply to msg c3e91cdf1f27407ca7151caef88750b1 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 947.213536] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c3e91cdf1f27407ca7151caef88750b1 [ 947.557685] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 947.573116] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 947.573357] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 947.573525] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 947.649363] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 947.649598] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 947.649763] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 947.650123] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 947.650308] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 947.650542] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 947.667571] nova-conductor[54074]: DEBUG nova.quota [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Getting quotas for project f0685baf7056496280733eae476a8935. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 947.670546] nova-conductor[54074]: DEBUG nova.quota [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Getting quotas for user 68a934a299dc48b0a7fc35f93d79db99 and project f0685baf7056496280733eae476a8935. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 947.676050] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] [instance: 37af2c2e-9c4e-445d-b128-c4c9137e73ca] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 947.676494] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 947.676786] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 947.676963] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 947.686629] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] [instance: 37af2c2e-9c4e-445d-b128-c4c9137e73ca] block_device_mapping [BlockDeviceMapping(attachment_id=535aa4c1-4912-40c0-9e8a-36fdb52c5f68,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='63e98c38-5d45-4f38-ba12-8536414be71d',volume_size=1,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 947.687478] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 947.687765] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 947.688096] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 947.706839] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 947.706839] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 947.706839] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 948.435425] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=394,nova_cell1:UPDATE=202,nova_cell1:INSERT=33,nova_cell1:SAVEPOINT=33,nova_cell1:RELEASE=33 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 950.916717] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=2 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 950.954750] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=10,nova_cell1:UPDATE=3,nova_cell1:INSERT=16,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 950.968195] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=14,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 950.976465] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=12,nova_api:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 953.884420] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] [instance: 5f58014c-e132-4fad-9ba7-bc183318200f] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 66844096-b3f6-4317-9dfc-3d1d1cd65bd5, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 5f58014c-e132-4fad-9ba7-bc183318200f was re-scheduled: Binding failed for port 66844096-b3f6-4317-9dfc-3d1d1cd65bd5, please check neutron logs for more information.\n'] [ 953.884982] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 953.885201] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5f58014c-e132-4fad-9ba7-bc183318200f.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5f58014c-e132-4fad-9ba7-bc183318200f. [ 953.885509] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] [instance: 5f58014c-e132-4fad-9ba7-bc183318200f] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5f58014c-e132-4fad-9ba7-bc183318200f. [ 953.904770] nova-conductor[54074]: DEBUG nova.network.neutron [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] [instance: 5f58014c-e132-4fad-9ba7-bc183318200f] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 953.921291] nova-conductor[54074]: DEBUG nova.network.neutron [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] [instance: 5f58014c-e132-4fad-9ba7-bc183318200f] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 953.924013] nova-conductor[54074]: DEBUG nova.network.neutron [None req-8c2f415b-a434-4212-a156-3c4cb8cd605b tempest-ServerRescueTestJSON-604246021 tempest-ServerRescueTestJSON-604246021-project-member] [instance: 5f58014c-e132-4fad-9ba7-bc183318200f] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 957.663707] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 957.690500] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=3,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=8,nova_cell1:RELEASE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 957.700501] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=6,nova_api:UPDATE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 957.702853] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=6,nova_api:DELETE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 962.320994] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] [instance: 8bd1a8aa-844b-47ca-9296-0c30af695984] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 803e3dd5-6dbb-437e-931c-d3616c54eafb, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 8bd1a8aa-844b-47ca-9296-0c30af695984 was re-scheduled: Binding failed for port 803e3dd5-6dbb-437e-931c-d3616c54eafb, please check neutron logs for more information.\n'] [ 962.321548] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 962.321823] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8bd1a8aa-844b-47ca-9296-0c30af695984.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8bd1a8aa-844b-47ca-9296-0c30af695984. [ 962.321973] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] [instance: 8bd1a8aa-844b-47ca-9296-0c30af695984] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8bd1a8aa-844b-47ca-9296-0c30af695984. [ 962.340035] nova-conductor[54074]: DEBUG nova.network.neutron [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] [instance: 8bd1a8aa-844b-47ca-9296-0c30af695984] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 962.353654] nova-conductor[54074]: DEBUG nova.network.neutron [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] [instance: 8bd1a8aa-844b-47ca-9296-0c30af695984] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 962.356114] nova-conductor[54074]: DEBUG nova.network.neutron [None req-8bd3e89f-4adc-4738-9e63-c6a5798c33ed tempest-ServerTagsTestJSON-1874445576 tempest-ServerTagsTestJSON-1874445576-project-member] [instance: 8bd1a8aa-844b-47ca-9296-0c30af695984] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 964.013825] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 27fed863-1e27-4258-8b43-b8cd23e3c1c0] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 09fb75f7-ed62-435c-bde1-9bcbab653d49, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 27fed863-1e27-4258-8b43-b8cd23e3c1c0 was re-scheduled: Binding failed for port 09fb75f7-ed62-435c-bde1-9bcbab653d49, please check neutron logs for more information.\n'] [ 964.014839] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 964.015075] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 27fed863-1e27-4258-8b43-b8cd23e3c1c0.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 27fed863-1e27-4258-8b43-b8cd23e3c1c0. [ 964.015356] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 27fed863-1e27-4258-8b43-b8cd23e3c1c0] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 27fed863-1e27-4258-8b43-b8cd23e3c1c0. [ 964.036688] nova-conductor[54074]: DEBUG nova.network.neutron [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 27fed863-1e27-4258-8b43-b8cd23e3c1c0] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 964.058457] nova-conductor[54074]: DEBUG nova.network.neutron [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 27fed863-1e27-4258-8b43-b8cd23e3c1c0] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 964.061440] nova-conductor[54074]: DEBUG nova.network.neutron [None req-0a5a4312-162f-4102-bb0b-f6b8f1f79df9 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 27fed863-1e27-4258-8b43-b8cd23e3c1c0] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 965.406328] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Expecting reply to msg fa7735a21dc544c0b2b0d0686e7f907d in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 965.530371] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fa7735a21dc544c0b2b0d0686e7f907d [ 965.715918] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] [instance: 228d2a6d-6c16-472c-9326-2e4576d9648c] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port ff125b21-2e96-4688-8b92-5ff1f1891e83, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 228d2a6d-6c16-472c-9326-2e4576d9648c was re-scheduled: Binding failed for port ff125b21-2e96-4688-8b92-5ff1f1891e83, please check neutron logs for more information.\n'] [ 965.716602] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 965.716838] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 228d2a6d-6c16-472c-9326-2e4576d9648c.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 228d2a6d-6c16-472c-9326-2e4576d9648c. [ 965.717051] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] [instance: 228d2a6d-6c16-472c-9326-2e4576d9648c] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 228d2a6d-6c16-472c-9326-2e4576d9648c. [ 965.749016] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Expecting reply to msg 530996b7d4704282882d7038a2b52027 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 965.752634] nova-conductor[54074]: DEBUG nova.network.neutron [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] [instance: 228d2a6d-6c16-472c-9326-2e4576d9648c] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 965.772141] nova-conductor[54074]: DEBUG nova.network.neutron [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] [instance: 228d2a6d-6c16-472c-9326-2e4576d9648c] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 965.775839] nova-conductor[54074]: DEBUG nova.network.neutron [None req-44cc891d-a696-452c-824e-969d4e3bfe35 tempest-InstanceActionsV221TestJSON-1207156857 tempest-InstanceActionsV221TestJSON-1207156857-project-member] [instance: 228d2a6d-6c16-472c-9326-2e4576d9648c] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 965.867939] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 530996b7d4704282882d7038a2b52027 [ 965.910064] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 965.922478] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 965.922701] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 965.922869] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 965.953320] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 965.953556] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 965.953808] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 965.954094] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 965.954311] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 965.954478] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 965.963246] nova-conductor[54075]: DEBUG nova.quota [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Getting quotas for project 71a669cc62964b05bdaa71442c08a566. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 965.966049] nova-conductor[54075]: DEBUG nova.quota [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Getting quotas for user 2d003a5943d141cc9165e43148dec381 and project 71a669cc62964b05bdaa71442c08a566. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 965.971929] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 138f2594-adbe-4ce2-a395-40fae312981b] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 965.972460] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 965.972662] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 965.972825] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 965.976481] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 138f2594-adbe-4ce2-a395-40fae312981b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 965.977083] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 965.977277] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 965.977437] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 965.992646] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 965.992839] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 965.993616] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 966.255727] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 966.270804] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 966.271038] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 966.271207] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 966.309277] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 966.309462] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 966.309630] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 966.309997] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 966.310179] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 966.310336] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 966.318813] nova-conductor[54074]: DEBUG nova.quota [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Getting quotas for project f580a63417834f22aae91b47303194c0. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 966.321506] nova-conductor[54074]: DEBUG nova.quota [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Getting quotas for user 781023c2e5404ad98a48d9c2b5fc0f36 and project f580a63417834f22aae91b47303194c0. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 966.326253] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] [instance: b26c8ba7-e41f-4ebd-a7f5-da7ae1b03308] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 966.326661] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 966.326855] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 966.327018] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 966.330114] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] [instance: b26c8ba7-e41f-4ebd-a7f5-da7ae1b03308] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 966.330721] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 966.330913] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 966.331072] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 966.352487] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 966.352754] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 966.352967] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-435e61e1-1cc4-4f2c-a279-d678d754622b tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 968.447916] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: a879b81d-fb5a-483b-9c2a-4a5c416c1caa] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port c678b72f-ea04-4809-afb1-fe2f0b013d2b, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance a879b81d-fb5a-483b-9c2a-4a5c416c1caa was re-scheduled: Binding failed for port c678b72f-ea04-4809-afb1-fe2f0b013d2b, please check neutron logs for more information.\n'] [ 968.448897] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 968.449036] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a879b81d-fb5a-483b-9c2a-4a5c416c1caa.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a879b81d-fb5a-483b-9c2a-4a5c416c1caa. [ 968.449325] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: a879b81d-fb5a-483b-9c2a-4a5c416c1caa] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a879b81d-fb5a-483b-9c2a-4a5c416c1caa. [ 968.474836] nova-conductor[54074]: DEBUG nova.network.neutron [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: a879b81d-fb5a-483b-9c2a-4a5c416c1caa] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 968.500778] nova-conductor[54074]: DEBUG nova.network.neutron [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: a879b81d-fb5a-483b-9c2a-4a5c416c1caa] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 968.504809] nova-conductor[54074]: DEBUG nova.network.neutron [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: a879b81d-fb5a-483b-9c2a-4a5c416c1caa] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 970.198679] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 9ab3a71f-7e26-4d29-b006-6dbebcee16e1] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 977b3bb3-ccfc-4735-af64-154240ffc670, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 9ab3a71f-7e26-4d29-b006-6dbebcee16e1 was re-scheduled: Binding failed for port 977b3bb3-ccfc-4735-af64-154240ffc670, please check neutron logs for more information.\n'] [ 970.198679] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 970.198679] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9ab3a71f-7e26-4d29-b006-6dbebcee16e1.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9ab3a71f-7e26-4d29-b006-6dbebcee16e1. [ 970.198679] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 9ab3a71f-7e26-4d29-b006-6dbebcee16e1] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9ab3a71f-7e26-4d29-b006-6dbebcee16e1. [ 970.233985] nova-conductor[54074]: DEBUG nova.network.neutron [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 9ab3a71f-7e26-4d29-b006-6dbebcee16e1] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 970.253542] nova-conductor[54074]: DEBUG nova.network.neutron [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 9ab3a71f-7e26-4d29-b006-6dbebcee16e1] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 970.259637] nova-conductor[54074]: DEBUG nova.network.neutron [None req-34b71be8-5b92-4991-982d-bb3cebe6c941 tempest-MultipleCreateTestJSON-1958664483 tempest-MultipleCreateTestJSON-1958664483-project-member] [instance: 9ab3a71f-7e26-4d29-b006-6dbebcee16e1] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 970.857237] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Expecting reply to msg ffda6a75362541d582ff5a25e1d838af in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 970.992842] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ffda6a75362541d582ff5a25e1d838af [ 971.360080] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 971.372439] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 971.372678] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 971.372850] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 971.399626] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 971.399851] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 971.400028] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 971.400485] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 971.400695] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 971.400854] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 971.410691] nova-conductor[54074]: DEBUG nova.quota [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Getting quotas for project de0340ebdce94ded876d6eb2769afbcc. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 971.413363] nova-conductor[54074]: DEBUG nova.quota [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Getting quotas for user 8d514402d01f44c59eabf4ae0b14103c and project de0340ebdce94ded876d6eb2769afbcc. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 971.417937] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] [instance: 121dbfda-87d9-4733-a7d2-3ffa6f54df36] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 971.418357] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 971.418550] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 971.418710] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 971.422578] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] [instance: 121dbfda-87d9-4733-a7d2-3ffa6f54df36] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 971.423176] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 971.423367] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 971.423528] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 971.439009] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 971.439203] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 971.439371] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 973.620718] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: b04cc451-a497-474f-90dd-282a469ff3c2] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 73a1a0e1-a2b2-4bfd-af75-9650be5f9837, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance b04cc451-a497-474f-90dd-282a469ff3c2 was re-scheduled: Binding failed for port 73a1a0e1-a2b2-4bfd-af75-9650be5f9837, please check neutron logs for more information.\n'] [ 973.621347] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 973.621623] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b04cc451-a497-474f-90dd-282a469ff3c2.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b04cc451-a497-474f-90dd-282a469ff3c2. [ 973.621938] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: b04cc451-a497-474f-90dd-282a469ff3c2] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b04cc451-a497-474f-90dd-282a469ff3c2. [ 973.646442] nova-conductor[54075]: DEBUG nova.network.neutron [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: b04cc451-a497-474f-90dd-282a469ff3c2] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 973.750843] nova-conductor[54075]: DEBUG nova.network.neutron [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: b04cc451-a497-474f-90dd-282a469ff3c2] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 973.753986] nova-conductor[54075]: DEBUG nova.network.neutron [None req-ceb8b7a2-1f2f-4640-aa90-301c645b6da0 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: b04cc451-a497-474f-90dd-282a469ff3c2] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 975.958474] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 975.979319] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=3,nova_cell1:INSERT=7 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 975.988604] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=7 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 975.990594] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=5,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 976.276550] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Expecting reply to msg b790ed8a75ef412ea5b1c2d090119449 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 976.399975] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b790ed8a75ef412ea5b1c2d090119449 [ 976.780391] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 976.792837] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 976.793331] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 976.793626] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 976.822951] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 976.823474] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 976.823861] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 976.824402] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 976.824706] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 976.824999] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 976.833601] nova-conductor[54075]: DEBUG nova.quota [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Getting quotas for project 98934316e57a4ea69b2bb5a2f2aaf251. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 976.836517] nova-conductor[54075]: DEBUG nova.quota [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Getting quotas for user 2883d8b8ab764050a13c8b3a56318c34 and project 98934316e57a4ea69b2bb5a2f2aaf251. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 976.841548] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: 2e484358-0037-41b0-bf66-534fc7116d34] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 976.842075] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 976.842416] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 976.842696] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 976.846834] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: 2e484358-0037-41b0-bf66-534fc7116d34] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 976.847626] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 976.848042] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 976.848337] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 976.862733] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 976.863161] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 976.863491] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 977.547786] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: d622ca94-7f5c-47f4-8077-ff37f64eea02] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 78639c1d-5491-49ff-8e1b-f3202998b190, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance d622ca94-7f5c-47f4-8077-ff37f64eea02 was re-scheduled: Binding failed for port 78639c1d-5491-49ff-8e1b-f3202998b190, please check neutron logs for more information.\n'] [ 977.548428] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 977.548660] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d622ca94-7f5c-47f4-8077-ff37f64eea02.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d622ca94-7f5c-47f4-8077-ff37f64eea02. [ 977.548964] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: d622ca94-7f5c-47f4-8077-ff37f64eea02] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d622ca94-7f5c-47f4-8077-ff37f64eea02. [ 977.570681] nova-conductor[54074]: DEBUG nova.network.neutron [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: d622ca94-7f5c-47f4-8077-ff37f64eea02] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 977.584255] nova-conductor[54074]: DEBUG nova.network.neutron [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: d622ca94-7f5c-47f4-8077-ff37f64eea02] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 977.587907] nova-conductor[54074]: DEBUG nova.network.neutron [None req-fa20862a-8aee-4891-b61f-862bd7eaa9b5 tempest-ServerDiskConfigTestJSON-32558757 tempest-ServerDiskConfigTestJSON-32558757-project-member] [instance: d622ca94-7f5c-47f4-8077-ff37f64eea02] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 979.239851] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] [instance: 49a0249a-f322-47f6-b723-2af2b701902c] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port bc6eae64-58f5-45f8-84e4-a333eeaf85e1, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 49a0249a-f322-47f6-b723-2af2b701902c was re-scheduled: Binding failed for port bc6eae64-58f5-45f8-84e4-a333eeaf85e1, please check neutron logs for more information.\n'] [ 979.239851] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 979.239851] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 49a0249a-f322-47f6-b723-2af2b701902c.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 49a0249a-f322-47f6-b723-2af2b701902c. [ 979.239851] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] [instance: 49a0249a-f322-47f6-b723-2af2b701902c] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 49a0249a-f322-47f6-b723-2af2b701902c. [ 979.260195] nova-conductor[54075]: DEBUG nova.network.neutron [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] [instance: 49a0249a-f322-47f6-b723-2af2b701902c] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 979.281366] nova-conductor[54075]: DEBUG nova.network.neutron [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] [instance: 49a0249a-f322-47f6-b723-2af2b701902c] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 979.284529] nova-conductor[54075]: DEBUG nova.network.neutron [None req-c78553e6-1311-4294-a4ff-f2ae1769ca0a tempest-ServersTestFqdnHostnames-1289654912 tempest-ServersTestFqdnHostnames-1289654912-project-member] [instance: 49a0249a-f322-47f6-b723-2af2b701902c] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 980.976340] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: f9cb0f9d-fcc7-4241-b6db-ce1cf8f3ed54] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 886b872c-e114-45cf-8809-f9755f25bde6, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance f9cb0f9d-fcc7-4241-b6db-ce1cf8f3ed54 was re-scheduled: Binding failed for port 886b872c-e114-45cf-8809-f9755f25bde6, please check neutron logs for more information.\n'] [ 980.976340] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 980.976340] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f9cb0f9d-fcc7-4241-b6db-ce1cf8f3ed54.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f9cb0f9d-fcc7-4241-b6db-ce1cf8f3ed54. [ 980.976340] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: f9cb0f9d-fcc7-4241-b6db-ce1cf8f3ed54] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f9cb0f9d-fcc7-4241-b6db-ce1cf8f3ed54. [ 981.000856] nova-conductor[54074]: DEBUG nova.network.neutron [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: f9cb0f9d-fcc7-4241-b6db-ce1cf8f3ed54] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 981.023022] nova-conductor[54074]: DEBUG nova.network.neutron [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: f9cb0f9d-fcc7-4241-b6db-ce1cf8f3ed54] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 981.026536] nova-conductor[54074]: DEBUG nova.network.neutron [None req-9c125533-59f7-41a5-9da8-a2dad34bc0fb tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: f9cb0f9d-fcc7-4241-b6db-ce1cf8f3ed54] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 981.406414] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 981.425606] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=6,nova_cell1:SAVEPOINT=2,nova_cell1:INSERT=16,nova_cell1:RELEASE=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 981.436101] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=13 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 981.437937] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=11,nova_api:UPDATE=2,nova_api:DELETE=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 982.722996] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Expecting reply to msg ed2da11c6151471b96a87a3ee390e481 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 982.848688] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ed2da11c6151471b96a87a3ee390e481 [ 983.225330] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 983.238731] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 983.239000] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 983.239377] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 983.272050] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 983.272336] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 983.272531] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 983.272928] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 983.273129] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 983.273311] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 983.282403] nova-conductor[54074]: DEBUG nova.quota [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Getting quotas for project aa59611b27bb41beb282e68ccfa6fadc. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 983.285380] nova-conductor[54074]: DEBUG nova.quota [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Getting quotas for user ba5969be1a254281b4dffd81aa84ae7e and project aa59611b27bb41beb282e68ccfa6fadc. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 983.290672] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 31c90ad4-71c9-4fea-b548-a8d0e8ff56bf] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 983.291142] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 983.291360] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 983.291551] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 983.295124] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 31c90ad4-71c9-4fea-b548-a8d0e8ff56bf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 983.295999] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 983.295999] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 983.296185] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 983.314118] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 983.314414] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 983.314613] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 983.425530] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Expecting reply to msg 0ae77be13ae448fd88259446e442c924 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 983.559161] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0ae77be13ae448fd88259446e442c924 [ 983.935605] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 983.953891] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 983.954121] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 983.954276] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 983.986370] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 983.986580] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 983.986740] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 983.987090] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 983.987266] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 983.987418] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 983.995845] nova-conductor[54074]: DEBUG nova.quota [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Getting quotas for project f97e7c0258f74b54aeed6c19db7c8cfe. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 983.997932] nova-conductor[54074]: DEBUG nova.quota [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Getting quotas for user 299130133b5546bdb3dec4e7203533db and project f97e7c0258f74b54aeed6c19db7c8cfe. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 984.002939] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] [instance: 681f4b00-1ed8-47fb-9117-aa3745096e66] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 984.003889] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 984.004132] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 984.004307] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 984.009106] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] [instance: 681f4b00-1ed8-47fb-9117-aa3745096e66] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 984.009758] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 984.009958] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 984.010233] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 984.023411] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 984.023631] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 984.023797] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 986.828950] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 986.850610] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=3,nova_cell1:INSERT=7 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 986.858401] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=5 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 986.860523] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=7,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 989.518617] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-eff708e3-eb06-4078-b77f-334b03126925 tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Targeting cell 89980bcd-ea65-4338-957c-d9d250569837(cell1) for conductor method rebuild_instance {{(pid=54075) wrapper /opt/stack/nova/nova/conductor/manager.py:95}} [ 989.519062] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-eff708e3-eb06-4078-b77f-334b03126925 tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 989.519315] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-eff708e3-eb06-4078-b77f-334b03126925 tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 989.519485] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-eff708e3-eb06-4078-b77f-334b03126925 tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 989.527739] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-eff708e3-eb06-4078-b77f-334b03126925 tempest-ServerShowV257Test-219043525 tempest-ServerShowV257Test-219043525-project-member] [instance: b26c8ba7-e41f-4ebd-a7f5-da7ae1b03308] No migration record for the rebuild/evacuate request. {{(pid=54075) rebuild_instance /opt/stack/nova/nova/conductor/manager.py:1227}} [ 991.985223] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] [instance: 37af2c2e-9c4e-445d-b128-c4c9137e73ca] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 3e6eec16-c463-49ba-b551-ce1647747aad, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 37af2c2e-9c4e-445d-b128-c4c9137e73ca was re-scheduled: Binding failed for port 3e6eec16-c463-49ba-b551-ce1647747aad, please check neutron logs for more information.\n'] [ 991.985821] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 991.986107] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 37af2c2e-9c4e-445d-b128-c4c9137e73ca.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 37af2c2e-9c4e-445d-b128-c4c9137e73ca. [ 991.986473] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] [instance: 37af2c2e-9c4e-445d-b128-c4c9137e73ca] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 37af2c2e-9c4e-445d-b128-c4c9137e73ca. [ 992.009849] nova-conductor[54074]: DEBUG nova.network.neutron [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] [instance: 37af2c2e-9c4e-445d-b128-c4c9137e73ca] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 992.028471] nova-conductor[54074]: DEBUG nova.network.neutron [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] [instance: 37af2c2e-9c4e-445d-b128-c4c9137e73ca] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 992.031662] nova-conductor[54074]: DEBUG nova.network.neutron [None req-ca8558f0-4a71-434d-b657-9aaba1e21650 tempest-ServerActionsV293TestJSON-1520280389 tempest-ServerActionsV293TestJSON-1520280389-project-member] [instance: 37af2c2e-9c4e-445d-b128-c4c9137e73ca] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 993.994565] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 994.013131] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=6,nova_cell1:INSERT=15,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 994.020084] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=11,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 994.021787] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 998.109002] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 138f2594-adbe-4ce2-a395-40fae312981b] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port d6902f58-e935-4bf3-9383-425e88f02b8a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 138f2594-adbe-4ce2-a395-40fae312981b was re-scheduled: Binding failed for port d6902f58-e935-4bf3-9383-425e88f02b8a, please check neutron logs for more information.\n'] [ 998.109981] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 998.110323] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 138f2594-adbe-4ce2-a395-40fae312981b.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 138f2594-adbe-4ce2-a395-40fae312981b. [ 998.110711] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 138f2594-adbe-4ce2-a395-40fae312981b] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 138f2594-adbe-4ce2-a395-40fae312981b. [ 998.131307] nova-conductor[54074]: DEBUG nova.network.neutron [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 138f2594-adbe-4ce2-a395-40fae312981b] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 998.148863] nova-conductor[54074]: DEBUG nova.network.neutron [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 138f2594-adbe-4ce2-a395-40fae312981b] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 998.151967] nova-conductor[54074]: DEBUG nova.network.neutron [None req-9142db86-db4a-470d-bee7-746e13ac7a7c tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 138f2594-adbe-4ce2-a395-40fae312981b] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 999.478195] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Expecting reply to msg 5e6ad6952e894056a633e252c6f8a142 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 999.533368] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=2 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 999.545298] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=4,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 999.594379] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5e6ad6952e894056a633e252c6f8a142 [ 999.816731] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] [instance: 121dbfda-87d9-4733-a7d2-3ffa6f54df36] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 9f70bcb1-71b1-4cc5-8744-c080b2082474, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 121dbfda-87d9-4733-a7d2-3ffa6f54df36 was re-scheduled: Binding failed for port 9f70bcb1-71b1-4cc5-8744-c080b2082474, please check neutron logs for more information.\n'] [ 999.817162] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 999.817162] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 121dbfda-87d9-4733-a7d2-3ffa6f54df36.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 121dbfda-87d9-4733-a7d2-3ffa6f54df36. [ 999.817632] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] [instance: 121dbfda-87d9-4733-a7d2-3ffa6f54df36] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 121dbfda-87d9-4733-a7d2-3ffa6f54df36. [ 999.843170] nova-conductor[54075]: DEBUG nova.network.neutron [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] [instance: 121dbfda-87d9-4733-a7d2-3ffa6f54df36] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 999.858870] nova-conductor[54075]: DEBUG nova.network.neutron [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] [instance: 121dbfda-87d9-4733-a7d2-3ffa6f54df36] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 999.864074] nova-conductor[54075]: DEBUG nova.network.neutron [None req-e0c8b7e4-7390-4487-864b-bd77985d49b7 tempest-ServersNegativeTestJSON-1356953462 tempest-ServersNegativeTestJSON-1356953462-project-member] [instance: 121dbfda-87d9-4733-a7d2-3ffa6f54df36] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 999.981025] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 999.993426] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 999.993659] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 999.993827] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1000.019236] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1000.019476] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1000.019629] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1000.020013] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1000.020202] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1000.020364] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1000.028893] nova-conductor[54074]: DEBUG nova.quota [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Getting quotas for project 71a669cc62964b05bdaa71442c08a566. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1000.031614] nova-conductor[54074]: DEBUG nova.quota [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Getting quotas for user 2d003a5943d141cc9165e43148dec381 and project 71a669cc62964b05bdaa71442c08a566. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1000.036340] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: f6587428-62b2-4e71-a585-4f794c96d04e] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1000.036747] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1000.036942] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1000.037107] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1000.040483] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: f6587428-62b2-4e71-a585-4f794c96d04e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1000.041146] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1000.041344] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1000.041507] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1000.055621] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1000.056059] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1000.056146] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1001.444562] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: 2e484358-0037-41b0-bf66-534fc7116d34] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 347cd73f-5ec1-45b9-9ed9-a242e25415ac, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 2e484358-0037-41b0-bf66-534fc7116d34 was re-scheduled: Binding failed for port 347cd73f-5ec1-45b9-9ed9-a242e25415ac, please check neutron logs for more information.\n'] [ 1001.445114] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1001.445333] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2e484358-0037-41b0-bf66-534fc7116d34.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2e484358-0037-41b0-bf66-534fc7116d34. [ 1001.445540] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: 2e484358-0037-41b0-bf66-534fc7116d34] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2e484358-0037-41b0-bf66-534fc7116d34. [ 1001.471136] nova-conductor[54075]: DEBUG nova.network.neutron [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: 2e484358-0037-41b0-bf66-534fc7116d34] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1001.487126] nova-conductor[54075]: DEBUG nova.network.neutron [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: 2e484358-0037-41b0-bf66-534fc7116d34] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1001.504518] nova-conductor[54075]: DEBUG nova.network.neutron [None req-e086231c-0ff8-43f9-92f7-bb0c4d4e59d8 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: 2e484358-0037-41b0-bf66-534fc7116d34] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1003.867250] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Expecting reply to msg e2a64a7a3b024f969c20d1e4d50664e6 in queue reply_7e6dfb74b2914bba9471d5b6164dc3bb [ 1003.983512] nova-conductor[54075]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e2a64a7a3b024f969c20d1e4d50664e6 [ 1004.369402] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54075) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1004.383281] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1004.383703] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1004.383991] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1004.413283] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1004.413682] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1004.413976] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1004.414444] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1004.414751] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1004.415022] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1004.422097] nova-conductor[54075]: DEBUG nova.quota [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Getting quotas for project 98934316e57a4ea69b2bb5a2f2aaf251. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1004.424679] nova-conductor[54075]: DEBUG nova.quota [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Getting quotas for user 2883d8b8ab764050a13c8b3a56318c34 and project 98934316e57a4ea69b2bb5a2f2aaf251. Resources: {'instances', 'cores', 'ram'} {{(pid=54075) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1004.429820] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: e34cf886-0f9b-4b9c-91c9-d04d9edb08e2] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54075) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1004.430324] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1004.430651] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1004.430930] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1004.436549] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: e34cf886-0f9b-4b9c-91c9-d04d9edb08e2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54075) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1004.437301] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1004.437615] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1004.437890] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1004.451266] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1004.451606] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1004.451884] nova-conductor[54075]: DEBUG oslo_concurrency.lockutils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54075) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1004.789788] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=389,nova_cell1:UPDATE=187,nova_cell1:INSERT=24,nova_cell1:SAVEPOINT=39,nova_cell1:RELEASE=39 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1006.337968] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 31c90ad4-71c9-4fea-b548-a8d0e8ff56bf] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 9a962537-f015-4cde-9316-5978f15b14e0, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 31c90ad4-71c9-4fea-b548-a8d0e8ff56bf was re-scheduled: Binding failed for port 9a962537-f015-4cde-9316-5978f15b14e0, please check neutron logs for more information.\n'] [ 1006.338598] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1006.338826] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 31c90ad4-71c9-4fea-b548-a8d0e8ff56bf.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 31c90ad4-71c9-4fea-b548-a8d0e8ff56bf. [ 1006.339145] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 31c90ad4-71c9-4fea-b548-a8d0e8ff56bf] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 31c90ad4-71c9-4fea-b548-a8d0e8ff56bf. [ 1006.360225] nova-conductor[54075]: DEBUG nova.network.neutron [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 31c90ad4-71c9-4fea-b548-a8d0e8ff56bf] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1006.374801] nova-conductor[54075]: DEBUG nova.network.neutron [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 31c90ad4-71c9-4fea-b548-a8d0e8ff56bf] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1006.378086] nova-conductor[54075]: DEBUG nova.network.neutron [None req-338b145e-959f-4ce0-811b-0dcff2db841b tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 31c90ad4-71c9-4fea-b548-a8d0e8ff56bf] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1007.805993] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Expecting reply to msg 62cf3ec40e444ef08bd9d526794ca7c0 in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 1007.924724] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 62cf3ec40e444ef08bd9d526794ca7c0 [ 1007.947673] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] [instance: 681f4b00-1ed8-47fb-9117-aa3745096e66] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 6ffa751a-5526-4526-875d-e0756fac6200, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 681f4b00-1ed8-47fb-9117-aa3745096e66 was re-scheduled: Binding failed for port 6ffa751a-5526-4526-875d-e0756fac6200, please check neutron logs for more information.\n'] [ 1007.948096] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1007.948337] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 681f4b00-1ed8-47fb-9117-aa3745096e66.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 681f4b00-1ed8-47fb-9117-aa3745096e66. [ 1007.948543] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] [instance: 681f4b00-1ed8-47fb-9117-aa3745096e66] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 681f4b00-1ed8-47fb-9117-aa3745096e66. [ 1007.978038] nova-conductor[54075]: DEBUG nova.network.neutron [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] [instance: 681f4b00-1ed8-47fb-9117-aa3745096e66] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1007.992791] nova-conductor[54075]: DEBUG nova.network.neutron [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] [instance: 681f4b00-1ed8-47fb-9117-aa3745096e66] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1007.995990] nova-conductor[54075]: DEBUG nova.network.neutron [None req-51943235-ad3a-49a7-a9b9-49ac95f4e3e4 tempest-ServerAddressesNegativeTestJSON-1097886150 tempest-ServerAddressesNegativeTestJSON-1097886150-project-member] [instance: 681f4b00-1ed8-47fb-9117-aa3745096e66] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1008.311066] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1008.321525] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1008.322069] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1008.322379] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1008.351819] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1008.352335] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1008.352648] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1008.353160] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1008.353491] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1008.353792] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1008.365623] nova-conductor[54074]: DEBUG nova.quota [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Getting quotas for project aa59611b27bb41beb282e68ccfa6fadc. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1008.368512] nova-conductor[54074]: DEBUG nova.quota [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Getting quotas for user ba5969be1a254281b4dffd81aa84ae7e and project aa59611b27bb41beb282e68ccfa6fadc. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1008.373644] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 8bfbf340-da8b-4ce7-8fb3-6804f7bbc600] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1008.374317] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1008.374659] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1008.374980] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1008.377547] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 8bfbf340-da8b-4ce7-8fb3-6804f7bbc600] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1008.378401] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1008.378735] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1008.379052] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1008.393515] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1008.393951] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1008.394285] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1008.566810] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:UPDATE=217,nova_cell1:SELECT=411,nova_cell1:SAVEPOINT=44,nova_cell1:RELEASE=44,nova_cell1:INSERT=30,nova_cell1:DELETE=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1014.419312] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell0:SELECT=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1014.440295] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=3,nova_cell1:INSERT=7 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1014.447094] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=6,nova_api:UPDATE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1014.450362] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_api:SELECT=6,nova_api:DELETE=1 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1018.173294] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: f6587428-62b2-4e71-a585-4f794c96d04e] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 6801707a-9500-4be7-9efe-705062339599, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance f6587428-62b2-4e71-a585-4f794c96d04e was re-scheduled: Binding failed for port 6801707a-9500-4be7-9efe-705062339599, please check neutron logs for more information.\n'] [ 1018.173860] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1018.174102] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f6587428-62b2-4e71-a585-4f794c96d04e.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f6587428-62b2-4e71-a585-4f794c96d04e. [ 1018.174312] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: f6587428-62b2-4e71-a585-4f794c96d04e] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f6587428-62b2-4e71-a585-4f794c96d04e. [ 1018.193946] nova-conductor[54075]: DEBUG nova.network.neutron [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: f6587428-62b2-4e71-a585-4f794c96d04e] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1018.214669] nova-conductor[54075]: DEBUG nova.network.neutron [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: f6587428-62b2-4e71-a585-4f794c96d04e] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1018.218725] nova-conductor[54075]: DEBUG nova.network.neutron [None req-c714cd2b-55ed-48ba-813e-5196605e1527 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: f6587428-62b2-4e71-a585-4f794c96d04e] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1018.357405] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1018.381374] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=6,nova_cell1:INSERT=14 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1018.389207] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=11 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1018.391015] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=2,nova_api:DELETE=2 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1019.163587] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Expecting reply to msg 7f0137b8957c4e808a2c7436a394c2db in queue reply_07d28983e5e94d2e89bfc63e924db1f9 [ 1019.289282] nova-conductor[54074]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7f0137b8957c4e808a2c7436a394c2db [ 1019.665837] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54074) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1019.678880] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1019.679096] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1019.679289] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1019.704290] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1019.704500] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1019.704671] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1019.705022] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1019.705243] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1019.705357] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1019.713222] nova-conductor[54074]: DEBUG nova.quota [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Getting quotas for project 71a669cc62964b05bdaa71442c08a566. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1019.715824] nova-conductor[54074]: DEBUG nova.quota [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Getting quotas for user 2d003a5943d141cc9165e43148dec381 and project 71a669cc62964b05bdaa71442c08a566. Resources: {'instances', 'cores', 'ram'} {{(pid=54074) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1019.720508] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 25797537-a81b-47de-8d65-afe0d154f317] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54074) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1019.720970] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1019.721167] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1019.721330] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1019.725567] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 25797537-a81b-47de-8d65-afe0d154f317] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4800b6ec-9841-4c82-b42e-97cce3beeec5',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54074) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1019.726149] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1019.726343] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1019.726506] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1019.746601] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Acquiring lock "89980bcd-ea65-4338-957c-d9d250569837" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1019.746806] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1019.746971] nova-conductor[54074]: DEBUG oslo_concurrency.lockutils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Lock "89980bcd-ea65-4338-957c-d9d250569837" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54074) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1020.264632] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: e34cf886-0f9b-4b9c-91c9-d04d9edb08e2] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 7ef816eb-00da-4479-b932-14b1601845cd, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance e34cf886-0f9b-4b9c-91c9-d04d9edb08e2 was re-scheduled: Binding failed for port 7ef816eb-00da-4479-b932-14b1601845cd, please check neutron logs for more information.\n'] [ 1020.265368] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1020.265755] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e34cf886-0f9b-4b9c-91c9-d04d9edb08e2.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e34cf886-0f9b-4b9c-91c9-d04d9edb08e2. [ 1020.266145] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: e34cf886-0f9b-4b9c-91c9-d04d9edb08e2] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e34cf886-0f9b-4b9c-91c9-d04d9edb08e2. [ 1020.295520] nova-conductor[54074]: DEBUG nova.network.neutron [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: e34cf886-0f9b-4b9c-91c9-d04d9edb08e2] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1020.310779] nova-conductor[54074]: DEBUG nova.network.neutron [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: e34cf886-0f9b-4b9c-91c9-d04d9edb08e2] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1020.313777] nova-conductor[54074]: DEBUG nova.network.neutron [None req-5d3b4d71-fa17-4478-afb6-149a36645019 tempest-AttachVolumeNegativeTest-726024558 tempest-AttachVolumeNegativeTest-726024558-project-member] [instance: e34cf886-0f9b-4b9c-91c9-d04d9edb08e2] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1022.937792] nova-conductor[54074]: ERROR nova.scheduler.utils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 8bfbf340-da8b-4ce7-8fb3-6804f7bbc600] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 8b395229-2200-402f-a201-d6593182bba4, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 8bfbf340-da8b-4ce7-8fb3-6804f7bbc600 was re-scheduled: Binding failed for port 8b395229-2200-402f-a201-d6593182bba4, please check neutron logs for more information.\n'] [ 1022.938449] nova-conductor[54074]: DEBUG nova.conductor.manager [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Rescheduling: True {{(pid=54074) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1022.938681] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8bfbf340-da8b-4ce7-8fb3-6804f7bbc600.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8bfbf340-da8b-4ce7-8fb3-6804f7bbc600. [ 1022.938888] nova-conductor[54074]: WARNING nova.scheduler.utils [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 8bfbf340-da8b-4ce7-8fb3-6804f7bbc600] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8bfbf340-da8b-4ce7-8fb3-6804f7bbc600. [ 1022.976959] nova-conductor[54074]: DEBUG nova.network.neutron [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 8bfbf340-da8b-4ce7-8fb3-6804f7bbc600] deallocate_for_instance() {{(pid=54074) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1022.991526] nova-conductor[54074]: DEBUG nova.network.neutron [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 8bfbf340-da8b-4ce7-8fb3-6804f7bbc600] Instance cache missing network info. {{(pid=54074) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1022.994956] nova-conductor[54074]: DEBUG nova.network.neutron [None req-1beace66-4f8d-4644-bd15-0ed816ec9aa5 tempest-ServersTestJSON-851829591 tempest-ServersTestJSON-851829591-project-member] [instance: 8bfbf340-da8b-4ce7-8fb3-6804f7bbc600] Updating instance_info_cache with network_info: [] {{(pid=54074) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1029.708445] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell0:SELECT=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1029.728476] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:SELECT=3,nova_cell1:INSERT=7 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1029.736224] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=6,nova_api:UPDATE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1029.740564] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_api:SELECT=6,nova_api:DELETE=1 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1032.419996] nova-conductor[54075]: ERROR nova.scheduler.utils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 25797537-a81b-47de-8d65-afe0d154f317] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 539, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 664, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port e35385bf-a80e-4160-96db-c2a15a6554ef, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 25797537-a81b-47de-8d65-afe0d154f317 was re-scheduled: Binding failed for port e35385bf-a80e-4160-96db-c2a15a6554ef, please check neutron logs for more information.\n'] [ 1032.420647] nova-conductor[54075]: DEBUG nova.conductor.manager [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Rescheduling: True {{(pid=54075) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1032.420878] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 25797537-a81b-47de-8d65-afe0d154f317.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 25797537-a81b-47de-8d65-afe0d154f317. [ 1032.421137] nova-conductor[54075]: WARNING nova.scheduler.utils [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 25797537-a81b-47de-8d65-afe0d154f317] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 25797537-a81b-47de-8d65-afe0d154f317. [ 1032.440756] nova-conductor[54075]: DEBUG nova.network.neutron [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 25797537-a81b-47de-8d65-afe0d154f317] deallocate_for_instance() {{(pid=54075) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 1032.455229] nova-conductor[54075]: DEBUG nova.network.neutron [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 25797537-a81b-47de-8d65-afe0d154f317] Instance cache missing network info. {{(pid=54075) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 1032.458149] nova-conductor[54075]: DEBUG nova.network.neutron [None req-222bfe20-21ee-4646-b8b7-c9761d55a2f6 tempest-DeleteServersTestJSON-377525300 tempest-DeleteServersTestJSON-377525300-project-member] [instance: 25797537-a81b-47de-8d65-afe0d154f317] Updating instance_info_cache with network_info: [] {{(pid=54075) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1042.428575] nova-conductor[54074]: DEBUG dbcounter [-] [54074] Writing DB stats nova_cell1:UPDATE=68,nova_cell1:SELECT=109,nova_cell1:DELETE=2,nova_cell1:SAVEPOINT=10,nova_cell1:RELEASE=10,nova_cell1:INSERT=5 {{(pid=54074) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1042.467985] nova-conductor[54075]: DEBUG dbcounter [-] [54075] Writing DB stats nova_cell1:SELECT=113,nova_cell1:SAVEPOINT=19,nova_cell1:RELEASE=19,nova_cell1:UPDATE=55,nova_cell1:INSERT=10 {{(pid=54075) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}}