[ 403.454783] nova-conductor[54011]: INFO dbcounter [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Registered counter for database nova_api [ 403.463596] nova-conductor[54011]: DEBUG oslo_db.sqlalchemy.engines [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=54011) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 403.492636] nova-conductor[54011]: DEBUG dbcounter [-] [54011] Writer thread running {{(pid=54011) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 403.494664] nova-conductor[54011]: DEBUG nova.context [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),8fb5f300-ab58-4d7c-afe8-8d0b4be88dee(cell1) {{(pid=54011) load_cells /opt/stack/nova/nova/context.py:464}} [ 403.496304] nova-conductor[54011]: DEBUG oslo_concurrency.lockutils [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54011) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 403.496537] nova-conductor[54011]: DEBUG oslo_concurrency.lockutils [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54011) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 403.496962] nova-conductor[54011]: DEBUG oslo_concurrency.lockutils [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54011) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 403.497373] nova-conductor[54011]: DEBUG oslo_concurrency.lockutils [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54011) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 403.497565] nova-conductor[54011]: DEBUG oslo_concurrency.lockutils [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54011) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 403.498498] nova-conductor[54011]: DEBUG oslo_concurrency.lockutils [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54011) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 403.499752] nova-conductor[54011]: INFO dbcounter [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Registered counter for database nova_cell0 [ 403.501966] nova-conductor[54011]: INFO dbcounter [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Registered counter for database nova_cell1 [ 403.504418] nova-conductor[54011]: DEBUG oslo_db.sqlalchemy.engines [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=54011) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 403.504907] nova-conductor[54011]: DEBUG oslo_db.sqlalchemy.engines [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=54011) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 403.585069] nova-conductor[54011]: DEBUG dbcounter [-] [54011] Writer thread running {{(pid=54011) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 403.587803] nova-conductor[54011]: DEBUG dbcounter [-] [54011] Writer thread running {{(pid=54011) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 403.589870] nova-conductor[54011]: DEBUG oslo_concurrency.lockutils [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Acquiring lock "singleton_lock" {{(pid=54011) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 403.590060] nova-conductor[54011]: DEBUG oslo_concurrency.lockutils [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Acquired lock "singleton_lock" {{(pid=54011) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 403.590302] nova-conductor[54011]: DEBUG oslo_concurrency.lockutils [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Releasing lock "singleton_lock" {{(pid=54011) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 403.590720] nova-conductor[54011]: INFO oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Starting 2 workers [ 403.595114] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Started child 54431 {{(pid=54011) _start_child /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/service.py:579}} [ 403.598496] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Started child 54432 {{(pid=54011) _start_child /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/service.py:579}} [ 403.598839] nova-conductor[54431]: INFO nova.service [-] Starting conductor node (version 0.0.1) [ 403.599196] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Full set of CONF: {{(pid=54011) wait /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/service.py:653}} [ 403.599396] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ******************************************************************************** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2806}} [ 403.599551] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Configuration options gathered from: {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2807}} [ 403.599729] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] command line args: ['--config-file', '/etc/nova/nova.conf'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2808}} [ 403.600046] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] config files: ['/etc/nova/nova.conf'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2809}} [ 403.600186] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ================================================================================ {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2811}} [ 403.600570] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] allow_resize_to_same_host = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.600781] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] arq_binding_timeout = 300 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.600979] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] block_device_allocate_retries = 60 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.601182] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] block_device_allocate_retries_interval = 3 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.601394] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cert = self.pem {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.601746] nova-conductor[54432]: INFO nova.service [-] Starting conductor node (version 0.0.1) [ 403.601923] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] compute_driver = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.602209] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] compute_monitors = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.602432] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] config_dir = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.602644] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] config_drive_format = iso9660 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.602795] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] config_file = ['/etc/nova/nova.conf'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.602994] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] config_source = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.603447] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] console_host = devstack {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.603620] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] control_exchange = nova {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.603825] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cpu_allocation_ratio = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.604017] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] daemon = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.604213] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] debug = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.604426] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] default_access_ip_network_name = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.604639] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] default_availability_zone = nova {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.604805] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] default_ephemeral_format = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.604993] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] default_green_pool_size = 1000 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.605330] nova-conductor[54431]: INFO dbcounter [None req-f4c7f4a3-004e-4dc1-a8f9-15ec1e0208e3 None None] Registered counter for database nova_cell1 [ 403.605511] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.605718] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] default_schedule_zone = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.605887] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] disk_allocation_ratio = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.606062] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] enable_new_services = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.606291] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] enabled_apis = ['osapi_compute'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.606493] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] enabled_ssl_apis = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.606697] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] flat_injected = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.606887] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] force_config_drive = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.607085] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] force_raw_images = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.607262] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] graceful_shutdown_timeout = 5 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.607430] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] heal_instance_info_cache_interval = 60 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.608008] nova-conductor[54432]: INFO dbcounter [None req-366bdbf5-7612-40aa-8d74-d6e9af80c647 None None] Registered counter for database nova_cell1 [ 403.608163] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] host = devstack {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.608399] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] initial_cpu_allocation_ratio = 4.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.608597] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] initial_disk_allocation_ratio = 1.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.608800] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] initial_ram_allocation_ratio = 1.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.609051] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.609236] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] instance_build_timeout = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.609418] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] instance_delete_interval = 300 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.609597] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] instance_format = [instance: %(uuid)s] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.609767] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] instance_name_template = instance-%08x {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.609929] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] instance_usage_audit = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.610135] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] instance_usage_audit_period = month {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.610336] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] instance_uuid_format = [instance: %(uuid)s] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.610516] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] instances_path = /opt/stack/data/nova/instances {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.610682] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] internal_service_availability_zone = internal {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.610836] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] key = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.611021] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] live_migration_retry_count = 30 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.611202] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] log_color = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.611387] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] log_config_append = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.611570] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.611734] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] log_dir = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.611960] nova-conductor[54431]: DEBUG oslo_db.sqlalchemy.engines [None req-f4c7f4a3-004e-4dc1-a8f9-15ec1e0208e3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=54431) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 403.612108] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] log_file = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.612254] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] log_options = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.612425] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] log_rotate_interval = 1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.612630] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] log_rotate_interval_type = days {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.612816] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] log_rotation_type = none {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.612954] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.613093] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.613266] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.613461] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.613589] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.613787] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] long_rpc_timeout = 1800 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.613962] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] max_concurrent_builds = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.614130] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] max_concurrent_live_migrations = 1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.614326] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] max_concurrent_snapshots = 5 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.614531] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] max_local_block_devices = 3 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.614720] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] max_logfile_count = 30 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.614881] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] max_logfile_size_mb = 200 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.615048] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] maximum_instance_delete_attempts = 5 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.615216] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] metadata_listen = 0.0.0.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.615428] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] metadata_listen_port = 8775 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.615600] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] metadata_workers = 2 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.615762] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] migrate_max_retries = -1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.615925] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] mkisofs_cmd = genisoimage {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.616161] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] my_block_storage_ip = 10.180.1.21 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.616296] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] my_ip = 10.180.1.21 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.616485] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] network_allocate_retries = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.616692] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.616858] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] osapi_compute_listen = 0.0.0.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.617026] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] osapi_compute_listen_port = 8774 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.617263] nova-conductor[54432]: DEBUG oslo_db.sqlalchemy.engines [None req-366bdbf5-7612-40aa-8d74-d6e9af80c647 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=54432) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 403.617408] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] osapi_compute_unique_server_name_scope = {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.617599] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] osapi_compute_workers = 2 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.617772] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] password_length = 12 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.617932] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] periodic_enable = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.618106] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] periodic_fuzzy_delay = 60 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.618276] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] pointer_model = usbtablet {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.618475] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] preallocate_images = none {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.618657] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] publish_errors = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.618786] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] pybasedir = /opt/stack/nova {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.618964] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ram_allocation_ratio = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.619154] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] rate_limit_burst = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.619330] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] rate_limit_except_level = CRITICAL {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.619505] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] rate_limit_interval = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.619680] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] reboot_timeout = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.619857] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] reclaim_instance_interval = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.620017] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] record = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.620178] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] reimage_timeout_per_gb = 20 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.620341] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] report_interval = 120 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.620501] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] rescue_timeout = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.620663] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] reserved_host_cpus = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.620816] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] reserved_host_disk_mb = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.620972] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] reserved_host_memory_mb = 512 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.621175] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] reserved_huge_pages = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.621337] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] resize_confirm_window = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.621515] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] resize_fs_using_block_device = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.621681] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] resume_guests_state_on_host_boot = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.621845] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.622033] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] rpc_response_timeout = 60 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.622196] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] run_external_periodic_tasks = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.622364] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] running_deleted_instance_action = reap {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.622539] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] running_deleted_instance_poll_interval = 1800 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.622731] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writer thread running {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 403.622874] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] running_deleted_instance_timeout = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.623067] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] scheduler_instance_sync_interval = 120 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.623241] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] service_down_time = 720 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.623443] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] servicegroup_driver = db {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.623608] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] shell_completion = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.623773] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] shelved_offload_time = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.623948] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] shelved_poll_interval = 3600 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.624139] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] shutdown_timeout = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.624312] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] source_is_ipv6 = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.624556] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ssl_only = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.624750] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] state_path = /opt/stack/data/nova {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.624916] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] sync_power_state_interval = 600 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.625088] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] sync_power_state_pool_size = 1000 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.625260] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] syslog_log_facility = LOG_USER {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.625501] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] tempdir = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.625777] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] timeout_nbd = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.626046] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] transport_url = **** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.626280] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] update_resources_interval = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.626509] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] use_cow_images = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.626779] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] use_eventlog = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.626960] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] use_journal = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.627138] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] use_json = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.627302] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] use_rootwrap_daemon = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.627461] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] use_stderr = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.627626] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] use_syslog = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.627785] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vcpu_pin_set = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.627974] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vif_plugging_is_fatal = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.628181] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vif_plugging_timeout = 300 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.628419] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] virt_mkfs = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.628606] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] volume_usage_poll_interval = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.628822] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writer thread running {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 403.628966] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] watch_log_file = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.629188] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] web = /usr/share/spice-html5 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2819}} [ 403.629494] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_concurrency.disable_process_locking = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.629699] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.629911] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.630092] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.630274] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_metrics.metrics_process_name = {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.630447] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.630615] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.630849] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.auth_strategy = keystone {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.631042] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.compute_link_prefix = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.631251] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.631467] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.dhcp_domain = novalocal {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.631778] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.enable_instance_password = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.632068] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.glance_link_prefix = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.632280] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.instance_list_cells_batch_fixed_size = 100 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.632499] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.instance_list_cells_batch_strategy = distributed {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.632682] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.instance_list_per_project_cells = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.633182] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.list_records_by_skipping_down_cells = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.633182] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.local_metadata_per_cell = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.633182] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.max_limit = 1000 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.633346] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.metadata_cache_expiration = 15 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.633517] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.neutron_default_tenant_id = default {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.633684] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.response_validation = warn {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.633853] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.use_neutron_default_nets = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.634026] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.vendordata_dynamic_connect_timeout = 5 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.634193] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.vendordata_dynamic_failure_fatal = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.634391] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.vendordata_dynamic_read_timeout = 5 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.634568] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.vendordata_dynamic_ssl_certfile = {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.634746] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.vendordata_dynamic_targets = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.634910] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.vendordata_jsonfile_path = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.635187] nova-conductor[54431]: DEBUG nova.service [None req-f4c7f4a3-004e-4dc1-a8f9-15ec1e0208e3 None None] Creating RPC server for service conductor {{(pid=54431) start /opt/stack/nova/nova/service.py:186}} [ 403.635338] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api.vendordata_providers = ['StaticJSON'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.635610] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.backend = dogpile.cache.memcached {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.635814] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.backend_argument = **** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.636012] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.config_prefix = cache.oslo {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.636230] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.dead_timeout = 60.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.636404] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.debug_cache_backend = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.636606] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.enable_retry_client = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.636777] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.enable_socket_keepalive = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.636954] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.enabled = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.637130] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.enforce_fips_mode = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.637326] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.expiration_time = 600 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.637513] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.hashclient_retry_attempts = 2 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.637766] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.hashclient_retry_delay = 1.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.637955] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.memcache_dead_retry = 300 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.638134] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.memcache_password = **** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.638309] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.638476] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.638649] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.memcache_pool_maxsize = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.638813] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.638981] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.memcache_sasl_enabled = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.639178] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.memcache_servers = ['localhost:11211'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.639350] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.memcache_socket_timeout = 1.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.639510] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.memcache_username = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.639680] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.proxies = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.639846] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.redis_db = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.640020] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.redis_password = **** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.640196] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.redis_sentinel_service_name = mymaster {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.640377] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.redis_sentinels = ['localhost:26379'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.640548] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.redis_server = localhost:6379 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.640713] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.redis_socket_timeout = 1.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.640879] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.redis_username = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.641055] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.retry_attempts = 2 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.641225] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.retry_delay = 0.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.641387] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.socket_keepalive_count = 1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.641553] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.socket_keepalive_idle = 1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.641718] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.socket_keepalive_interval = 1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.641874] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.tls_allowed_ciphers = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.642041] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.tls_cafile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.642204] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.tls_certfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.642403] nova-conductor[54432]: DEBUG nova.service [None req-366bdbf5-7612-40aa-8d74-d6e9af80c647 None None] Creating RPC server for service conductor {{(pid=54432) start /opt/stack/nova/nova/service.py:186}} [ 403.642548] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.tls_enabled = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.642734] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cache.tls_keyfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.642961] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cinder.auth_section = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.643182] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cinder.auth_type = password {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.643347] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cinder.cafile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.643542] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cinder.catalog_info = volumev3::publicURL {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.643712] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cinder.certfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.643875] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cinder.collect_timing = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.644063] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cinder.cross_az_attach = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.644235] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cinder.debug = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.644446] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cinder.endpoint_template = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.644618] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cinder.http_retries = 3 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.644781] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cinder.insecure = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.644940] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cinder.keyfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.645135] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cinder.os_region_name = RegionOne {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.645305] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cinder.split_loggers = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.645464] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cinder.timeout = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.645665] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] compute.consecutive_build_service_disable_threshold = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.645831] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] compute.cpu_dedicated_set = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.645993] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] compute.cpu_shared_set = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.646170] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] compute.image_type_exclude_list = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.646337] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] compute.live_migration_wait_for_vif_plug = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.646555] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] compute.max_concurrent_disk_ops = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.646700] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] compute.max_disk_devices_to_attach = -1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.646866] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.647049] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.647218] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] compute.resource_provider_association_refresh = 300 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.647382] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] compute.sharing_providers_max_uuids_per_request = 200 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.647547] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] compute.shutdown_retry_interval = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.647728] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] compute.vmdk_allowed_types = ['streamOptimized', 'monolithicSparse'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.647904] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] conductor.workers = 2 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.648090] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] console.allowed_origins = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.648251] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] console.ssl_ciphers = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.648424] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] console.ssl_minimum_version = default {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.648594] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] consoleauth.enforce_session_timeout = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.648794] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] consoleauth.token_ttl = 600 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.649021] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.cafile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.649183] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.certfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.649349] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.collect_timing = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.649514] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.connect_retries = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.649676] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.connect_retry_delay = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.649835] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.endpoint_override = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.649994] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.insecure = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.650168] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.keyfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.650330] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.max_version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.650488] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.min_version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.650647] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.region_name = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.650805] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.retriable_status_codes = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.650959] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.service_name = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.651140] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.service_type = accelerator {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.651310] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.split_loggers = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.651467] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.status_code_retries = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.651626] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.status_code_retry_delay = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.651784] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.timeout = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.651963] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.652145] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] cyborg.version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.652337] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.backend = sqlalchemy {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.652511] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.connection = **** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.652686] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.connection_debug = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.652858] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.connection_parameters = {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.653025] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.connection_recycle_time = 3600 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.653197] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.connection_trace = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.653360] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.db_inc_retry_interval = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.653523] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.db_max_retries = 20 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.653686] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.db_max_retry_interval = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.653844] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.db_retry_interval = 1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.654014] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.max_overflow = 50 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.654176] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.max_pool_size = 5 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.654360] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.max_retries = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.654549] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.mysql_sql_mode = TRADITIONAL {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.654713] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.mysql_wsrep_sync_wait = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.654870] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.pool_timeout = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.655043] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.retry_interval = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.655206] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.slave_connection = **** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.655366] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.sqlite_synchronous = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.655533] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] database.use_db_reconnect = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.655739] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.backend = sqlalchemy {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.655912] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.connection = **** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.656090] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.connection_debug = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.656260] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.connection_parameters = {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.656444] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.connection_recycle_time = 3600 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.656625] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.connection_trace = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.656788] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.db_inc_retry_interval = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.656952] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.db_max_retries = 20 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.657127] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.db_max_retry_interval = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.657291] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.db_retry_interval = 1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.657455] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.max_overflow = 50 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.657618] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.max_pool_size = 5 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.657776] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.max_retries = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.657941] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.mysql_sql_mode = TRADITIONAL {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.658111] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.mysql_wsrep_sync_wait = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.658272] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.pool_timeout = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.658431] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.retry_interval = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.658592] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.slave_connection = **** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.658748] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] api_database.sqlite_synchronous = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.658954] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] devices.enabled_mdev_types = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.659167] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.659344] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ephemeral_storage_encryption.default_format = luks {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.659510] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ephemeral_storage_encryption.enabled = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.659675] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ephemeral_storage_encryption.key_size = 512 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.659872] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.api_servers = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.660051] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.cafile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.660387] nova-conductor[54431]: DEBUG nova.service [None req-f4c7f4a3-004e-4dc1-a8f9-15ec1e0208e3 None None] Join ServiceGroup membership for this service conductor {{(pid=54431) start /opt/stack/nova/nova/service.py:203}} [ 403.660535] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.certfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.660716] nova-conductor[54431]: DEBUG nova.servicegroup.drivers.db [None req-f4c7f4a3-004e-4dc1-a8f9-15ec1e0208e3 None None] DB_Driver: join new ServiceGroup member devstack to the conductor group, service = {{(pid=54431) join /opt/stack/nova/nova/servicegroup/drivers/db.py:44}} [ 403.660854] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.collect_timing = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.661063] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.connect_retries = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.661232] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.connect_retry_delay = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.661418] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.debug = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.661591] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.default_trusted_certificate_ids = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.661753] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.enable_certificate_validation = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.661910] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.enable_rbd_download = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.662080] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.endpoint_override = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.662253] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.insecure = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.662558] nova-conductor[54432]: DEBUG nova.service [None req-366bdbf5-7612-40aa-8d74-d6e9af80c647 None None] Join ServiceGroup membership for this service conductor {{(pid=54432) start /opt/stack/nova/nova/service.py:203}} [ 403.662698] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.keyfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.662875] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.max_version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.663051] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.min_version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.663219] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.num_retries = 3 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.663392] nova-conductor[54432]: DEBUG nova.servicegroup.drivers.db [None req-366bdbf5-7612-40aa-8d74-d6e9af80c647 None None] DB_Driver: join new ServiceGroup member devstack to the conductor group, service = {{(pid=54432) join /opt/stack/nova/nova/servicegroup/drivers/db.py:44}} [ 403.663533] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.rbd_ceph_conf = {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.663716] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.rbd_connect_timeout = 5 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.663887] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.rbd_pool = {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.664088] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.rbd_user = {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.664258] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.region_name = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.664472] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.retriable_status_codes = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.664650] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.service_name = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.664826] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.service_type = image {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.664994] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.split_loggers = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.665163] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.status_code_retries = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.665323] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.status_code_retry_delay = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.665484] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.timeout = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.665660] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.valid_interfaces = ['internal', 'public'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.665826] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.verify_glance_signatures = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.665998] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] glance.version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.666180] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] guestfs.debug = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.666348] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] mks.enabled = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.666981] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.667243] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] image_cache.manager_interval = 2400 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.667445] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] image_cache.precache_concurrency = 1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.667645] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] image_cache.remove_unused_base_images = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.667823] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.667992] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.668186] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] image_cache.subdirectory_name = _base {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.668390] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.api_max_retries = 60 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.668558] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.api_retry_interval = 2 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.668719] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.auth_section = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.668902] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.auth_type = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.669071] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.cafile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.669231] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.certfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.669394] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.collect_timing = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.669577] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.conductor_group = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.669738] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.connect_retries = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.669891] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.connect_retry_delay = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.670058] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.endpoint_override = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.670226] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.insecure = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.670384] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.keyfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.670542] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.max_version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.670698] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.min_version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.670860] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.peer_list = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.671025] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.region_name = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.671185] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.retriable_status_codes = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.671349] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.serial_console_state_timeout = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.671507] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.service_name = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.671697] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.service_type = baremetal {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.671860] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.shard = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.672027] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.split_loggers = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.672193] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.status_code_retries = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.672351] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.status_code_retry_delay = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.672507] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.timeout = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.672688] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.valid_interfaces = ['internal', 'public'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.672853] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ironic.version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.673047] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.673271] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] key_manager.fixed_key = **** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.673492] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.673660] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican.barbican_api_version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.673821] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican.barbican_endpoint = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.674032] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican.barbican_endpoint_type = public {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.674230] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican.barbican_region_name = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.674413] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican.cafile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.674589] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican.certfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.674792] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican.collect_timing = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.674960] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican.insecure = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.675131] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican.keyfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.675319] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican.number_of_retries = 60 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.675483] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican.retry_delay = 1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.675650] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican.send_service_user_token = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.675811] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican.split_loggers = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.675965] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican.timeout = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.676132] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican.verify_ssl = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.676311] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican.verify_ssl_path = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.676499] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican_service_user.auth_section = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.676689] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican_service_user.auth_type = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.676849] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican_service_user.cafile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.677017] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican_service_user.certfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.677182] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican_service_user.collect_timing = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.677345] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican_service_user.insecure = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.677516] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican_service_user.keyfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.677709] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican_service_user.split_loggers = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.677870] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] barbican_service_user.timeout = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.678073] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vault.approle_role_id = **** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.678238] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vault.approle_secret_id = **** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.678411] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vault.kv_mountpoint = secret {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.678566] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vault.kv_path = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.678727] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vault.kv_version = 2 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.678904] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vault.namespace = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.679075] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vault.root_token_id = **** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.679251] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vault.ssl_ca_crt_file = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.679428] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vault.timeout = 60.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.679592] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vault.use_ssl = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.679781] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vault.vault_url = http://127.0.0.1:8200 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.679972] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.auth_section = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.680147] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.auth_type = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.680305] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.cafile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.680464] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.certfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.680625] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.collect_timing = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.680781] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.connect_retries = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.680940] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.connect_retry_delay = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.681107] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.endpoint_override = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.681273] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.insecure = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.681430] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.keyfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.681587] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.max_version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.681740] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.min_version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.681897] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.region_name = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.682066] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.retriable_status_codes = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.682227] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.service_name = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.682417] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.service_type = identity {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.682603] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.split_loggers = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.682763] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.status_code_retries = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.682922] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.status_code_retry_delay = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.683088] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.timeout = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.683273] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.valid_interfaces = ['internal', 'public'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.683433] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] keystone.version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.683678] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.connection_uri = {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.683866] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.cpu_mode = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.684043] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.cpu_model_extra_flags = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.684258] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.cpu_models = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.684462] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.cpu_power_governor_high = performance {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.684643] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.cpu_power_governor_low = powersave {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.684814] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.cpu_power_management = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.685014] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.cpu_power_management_strategy = cpu_state {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.685203] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.device_detach_attempts = 8 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.685369] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.device_detach_timeout = 20 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.685532] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.disk_cachemodes = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.685694] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.disk_prefix = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.685855] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.enabled_perf_events = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.686022] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.file_backed_memory = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.686188] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.gid_maps = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.686374] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.hw_disk_discard = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.686563] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.hw_machine_type = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.686737] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.images_rbd_ceph_conf = {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.686912] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.687102] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.687271] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.images_rbd_glance_store_name = {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.687441] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.images_rbd_pool = rbd {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.687611] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.images_type = default {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.687768] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.images_volume_group = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.687931] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.inject_key = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.688111] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.inject_partition = -2 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.688272] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.inject_password = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.688472] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.iscsi_iface = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.688637] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.iser_use_multipath = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.688796] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.live_migration_bandwidth = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.688955] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.live_migration_completion_timeout = 800 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.689136] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.live_migration_downtime = 500 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.689306] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.live_migration_downtime_delay = 75 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.689483] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.live_migration_downtime_steps = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.689646] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.live_migration_inbound_addr = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.689806] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.live_migration_permit_auto_converge = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.689966] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.live_migration_permit_post_copy = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.690137] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.live_migration_scheme = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.690313] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.live_migration_timeout_action = abort {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.690475] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.live_migration_tunnelled = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.690639] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.live_migration_uri = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.690799] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.live_migration_with_native_tls = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.690976] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.max_queues = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.691153] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.mem_stats_period_seconds = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.691414] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.migration_inbound_addr = 10.180.1.21 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.691596] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.nfs_mount_options = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.691825] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.691995] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.num_aoe_discover_tries = 3 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.692174] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.num_iser_scan_tries = 5 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.692334] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.num_memory_encrypted_guests = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.692500] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.num_nvme_discover_tries = 5 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.692662] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.num_pcie_ports = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.692826] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.num_volume_scan_tries = 5 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.692987] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.pmem_namespaces = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.693158] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.quobyte_client_cfg = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.693408] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.693579] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.rbd_connect_timeout = 5 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.693745] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.rbd_destroy_volume_retries = 12 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.693904] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.694073] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.rbd_secret_uuid = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.694229] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.rbd_user = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.694413] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.realtime_scheduler_priority = 1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.694619] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.remote_filesystem_transport = ssh {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.694783] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.rescue_image_id = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.694941] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.rescue_kernel_id = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.695111] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.rescue_ramdisk_id = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.695281] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.rng_dev_path = /dev/urandom {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.695461] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.rx_queue_size = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.695631] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.smbfs_mount_options = {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.695845] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.696024] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.snapshot_compression = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.696182] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.snapshot_image_format = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.696402] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.696616] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.sparse_logical_volumes = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.696799] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.swtpm_enabled = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.697033] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.swtpm_group = tss {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.697214] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.swtpm_user = tss {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.697403] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.sysinfo_serial = unique {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.697577] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.tb_cache_size = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.697733] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.tx_queue_size = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.697895] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.uid_maps = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.698071] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.use_virtio_for_bridges = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.698255] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.virt_type = kvm {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.698424] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.volume_clear = zero {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.698591] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.volume_clear_size = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.698756] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.volume_use_multipath = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.698913] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.vzstorage_cache_path = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.699092] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.699263] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.vzstorage_mount_group = qemu {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.699428] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.vzstorage_mount_opts = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.699594] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.vzstorage_mount_perms = 0770 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.699812] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.699979] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.vzstorage_mount_user = stack {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.700155] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] libvirt.wait_soft_reboot_seconds = 120 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.700350] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.auth_section = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.700521] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.auth_type = password {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.700682] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.cafile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.700840] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.certfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.701006] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.collect_timing = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.701194] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.connect_retries = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.701352] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.connect_retry_delay = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.701518] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.default_floating_pool = public {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.701676] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.endpoint_override = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.701835] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.extension_sync_interval = 600 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.701993] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.http_retries = 3 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.702168] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.insecure = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.702328] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.keyfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.702487] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.max_version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.702657] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.metadata_proxy_shared_secret = **** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.702810] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.min_version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.702974] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.ovs_bridge = br-int {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.703149] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.physnets = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.703314] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.region_name = RegionOne {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.703473] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.retriable_status_codes = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.703674] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.service_metadata_proxy = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.703838] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.service_name = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.704016] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.service_type = network {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.704188] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.split_loggers = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.704370] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.status_code_retries = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.704541] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.status_code_retry_delay = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.704725] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.timeout = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.704916] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.valid_interfaces = ['internal', 'public'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.705088] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] neutron.version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.705262] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] notifications.bdms_in_notifications = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.705433] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] notifications.default_level = INFO {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.705625] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] notifications.notification_format = unversioned {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.705786] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] notifications.notify_on_state_change = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.705958] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.706145] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] pci.alias = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.706315] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] pci.device_spec = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.706517] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] pci.report_in_placement = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.706723] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.auth_section = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.706897] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.auth_type = password {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.707094] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.auth_url = http://10.180.1.21/identity {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.707282] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.cafile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.707470] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.certfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.707641] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.collect_timing = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.707797] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.connect_retries = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.707955] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.connect_retry_delay = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.708145] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.default_domain_id = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.708303] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.default_domain_name = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.708478] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.domain_id = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.708637] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.domain_name = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.708793] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.endpoint_override = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.708955] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.insecure = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.709123] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.keyfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.709280] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.max_version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.709433] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.min_version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.709602] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.password = **** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.709757] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.project_domain_id = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.709918] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.project_domain_name = Default {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.710113] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.project_id = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.710287] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.project_name = service {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.710453] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.region_name = RegionOne {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.710618] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.retriable_status_codes = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.710771] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.service_name = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.710936] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.service_type = placement {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.711106] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.split_loggers = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.711263] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.status_code_retries = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.711420] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.status_code_retry_delay = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.711579] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.system_scope = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.711732] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.timeout = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.711887] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.trust_id = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.712053] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.user_domain_id = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.712224] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.user_domain_name = Default {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.712384] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.user_id = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.712555] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.username = nova {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.712736] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.valid_interfaces = ['internal', 'public'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.712894] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] placement.version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.713099] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] quota.cores = 20 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.713264] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] quota.count_usage_from_placement = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.713431] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] quota.driver = nova.quota.DbQuotaDriver {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.713616] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] quota.injected_file_content_bytes = 10240 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.713780] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] quota.injected_file_path_length = 255 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.713944] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] quota.injected_files = 5 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.714127] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] quota.instances = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.714296] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] quota.key_pairs = 100 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.714481] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] quota.metadata_items = 128 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.714926] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] quota.ram = 51200 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.714926] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] quota.recheck_quota = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.714982] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] quota.server_group_members = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.715140] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] quota.server_groups = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.715338] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.715527] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] scheduler.enable_isolated_aggregate_filtering = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.715710] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] scheduler.image_metadata_prefilter = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.715895] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.716068] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] scheduler.max_attempts = 3 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.716232] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] scheduler.max_placement_results = 1000 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.716449] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] scheduler.placement_aggregate_required_for_tenants = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.716615] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] scheduler.query_placement_for_image_type_support = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.716798] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.716968] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] scheduler.workers = 2 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.717181] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.717351] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.717579] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.717751] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.717914] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.718090] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.718305] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.718494] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.enabled_filters = ['ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.718664] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.host_subset_size = 1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.718825] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.hypervisor_version_weight_multiplier = 1.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.718998] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.image_properties_default_architecture = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.719174] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.719336] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.isolated_hosts = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.719498] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.isolated_images = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.719767] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.max_instances_per_host = 50 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.719811] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.max_io_ops_per_host = 8 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.719973] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.num_instances_weight_multiplier = 0.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.720142] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.pci_in_placement = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.720302] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.720464] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.720625] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.720781] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.720939] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.721111] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.721274] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.track_instance_changes = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.721449] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.721620] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] metrics.required = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.721779] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] metrics.weight_multiplier = 1.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.721960] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] metrics.weight_of_unavailable = -10000.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.722135] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] metrics.weight_setting = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.722461] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.722636] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] serial_console.enabled = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.722829] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] serial_console.port_range = 10000:20000 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.722998] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] serial_console.proxyclient_address = 127.0.0.1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.723178] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] serial_console.serialproxy_host = 0.0.0.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.723341] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] serial_console.serialproxy_port = 6083 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.723510] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] service_user.auth_section = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.723680] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] service_user.auth_type = password {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.723836] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] service_user.cafile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.723990] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] service_user.certfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.724167] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] service_user.collect_timing = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.724346] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] service_user.insecure = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.724521] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] service_user.keyfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.724697] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] service_user.send_service_user_token = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.724856] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] service_user.split_loggers = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.725018] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] service_user.timeout = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.725209] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] spice.agent_enabled = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.725369] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] spice.enabled = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.725707] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.725915] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] spice.html5proxy_host = 0.0.0.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.726143] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] spice.html5proxy_port = 6082 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.726306] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] spice.image_compression = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.726487] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] spice.jpeg_compression = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.726653] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] spice.playback_compression = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.726839] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] spice.require_secure = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.727013] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] spice.server_listen = 127.0.0.1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.727184] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] spice.server_proxyclient_address = 127.0.0.1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.727339] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] spice.streaming_mode = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.727498] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] spice.zlib_compression = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.727661] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] upgrade_levels.baseapi = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.727827] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] upgrade_levels.compute = auto {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.727982] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] upgrade_levels.conductor = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.728151] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] upgrade_levels.scheduler = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.728340] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vendordata_dynamic_auth.auth_section = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.728503] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vendordata_dynamic_auth.auth_type = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.728662] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vendordata_dynamic_auth.cafile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.728814] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vendordata_dynamic_auth.certfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.728969] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vendordata_dynamic_auth.collect_timing = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.729139] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vendordata_dynamic_auth.insecure = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.729297] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vendordata_dynamic_auth.keyfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.729474] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vendordata_dynamic_auth.split_loggers = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.729634] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vendordata_dynamic_auth.timeout = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.729838] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.api_retry_count = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.729996] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.ca_file = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.730162] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.cache_prefix = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.730314] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.cluster_name = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.730469] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.connection_pool_size = 10 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.730622] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.console_delay_seconds = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.730961] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.datastore_regex = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.730961] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.host_ip = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.731184] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.host_password = **** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.731240] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.host_port = 443 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.731387] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.host_username = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.731542] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.insecure = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.731693] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.integration_bridge = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.731849] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.maximum_objects = 100 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.732009] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.pbm_default_policy = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.732172] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.pbm_enabled = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.732325] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.pbm_wsdl_location = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.732491] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.732649] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.serial_port_proxy_uri = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.732805] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.serial_port_service_uri = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.732970] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.task_poll_interval = 0.5 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.733142] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.use_linked_clone = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.733309] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.vnc_keymap = en-us {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.733467] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.vnc_port = 5900 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.733626] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vmware.vnc_port_total = 10000 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.733840] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vnc.auth_schemes = ['none'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.734018] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vnc.enabled = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.734405] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.734606] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vnc.novncproxy_host = 0.0.0.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.734778] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vnc.novncproxy_port = 6080 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.734958] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vnc.server_listen = 127.0.0.1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.735148] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.735314] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vnc.vencrypt_ca_certs = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.735469] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vnc.vencrypt_client_cert = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.735633] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] vnc.vencrypt_client_key = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.735824] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.disable_compute_service_check_for_ffu = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.736010] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.disable_deep_image_inspection = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.736186] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.disable_fallback_pcpu_query = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.736349] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.disable_group_policy_check_upcall = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.736539] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.disable_libvirt_livesnapshot = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.736735] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.disable_rootwrap = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.736900] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.enable_numa_live_migration = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.737072] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.enable_qemu_monitor_announce_self = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.737236] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.737396] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.handle_virt_lifecycle_events = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.737579] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.libvirt_disable_apic = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.737750] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.never_download_image_if_on_rbd = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.737911] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.qemu_monitor_announce_self_count = 3 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.738078] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.qemu_monitor_announce_self_interval = 1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.738240] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.738399] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.skip_cpu_compare_at_startup = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.738560] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.skip_cpu_compare_on_dest = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.738722] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.738878] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.skip_reserve_in_use_ironic_nodes = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.739052] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.739209] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.739415] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.739589] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] wsgi.client_socket_timeout = 900 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.739752] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] wsgi.default_pool_size = 1000 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.739935] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] wsgi.keep_alive = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.740114] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] wsgi.max_header_line = 16384 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.740278] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] wsgi.secure_proxy_ssl_header = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.740439] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] wsgi.ssl_ca_file = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.740601] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] wsgi.ssl_cert_file = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.740762] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] wsgi.ssl_key_file = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.740921] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] wsgi.tcp_keepidle = 600 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.741124] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.741294] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] zvm.ca_file = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.741452] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] zvm.cloud_connector_url = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.741685] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.741851] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] zvm.reachable_timeout = 300 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.742088] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_policy.enforce_new_defaults = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.742584] nova-conductor[54011]: WARNING oslo_config.cfg [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] Deprecated: Option "enforce_scope" from group "oslo_policy" is deprecated for removal (This configuration was added temporarily to facilitate a smooth transition to the new RBAC. OpenStack will always enforce scope checks. This configuration option is deprecated and will be removed in the 2025.2 cycle.). Its value may be silently ignored in the future. [ 403.742774] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_policy.enforce_scope = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.742958] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_policy.policy_default_rule = default {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.743174] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_policy.policy_dirs = ['policy.d'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.743353] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_policy.policy_file = policy.yaml {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.743551] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.743718] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.743880] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.744049] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.744217] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.744424] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.744628] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.744855] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] profiler.connection_string = messaging:// {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.745057] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] profiler.enabled = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.745250] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] profiler.es_doc_type = notification {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.745434] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] profiler.es_scroll_size = 10000 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.745608] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] profiler.es_scroll_time = 2m {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.745770] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] profiler.filter_error_trace = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.745949] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] profiler.hmac_keys = **** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.746131] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] profiler.sentinel_service_name = mymaster {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.746319] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] profiler.socket_timeout = 0.1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.746509] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] profiler.trace_requests = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.746670] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] profiler.trace_sqlalchemy = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.746878] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] profiler_jaeger.process_tags = {} {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.747049] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] profiler_jaeger.service_name_prefix = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.747214] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] profiler_otlp.service_name_prefix = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.747409] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] remote_debug.host = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.747611] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] remote_debug.port = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.747825] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.747993] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.748171] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.748330] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.748493] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.748655] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.748822] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.748979] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.749147] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.749316] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.hostname = devstack {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.749474] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.749656] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.749912] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.750125] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.750300] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.processname = nova-conductor {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.750468] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.750629] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.750800] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.750957] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.751154] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.751325] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.751490] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.751657] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.751813] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.751968] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.752138] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.752300] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.752457] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.752624] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.752791] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.ssl = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.752959] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.ssl_ca_file = {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.753135] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.ssl_cert_file = {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.753296] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.753465] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.ssl_key_file = {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.753634] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.ssl_version = {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.753792] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_rabbit.use_queue_manager = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.753982] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.754176] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_notifications.retry = -1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.754357] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.754526] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_messaging_notifications.transport_url = **** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.754731] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.auth_section = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.754894] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.auth_type = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.755060] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.cafile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.755215] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.certfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.755375] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.collect_timing = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.755529] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.connect_retries = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.755707] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.connect_retry_delay = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.755863] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.endpoint_id = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.756022] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.endpoint_override = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.756204] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.insecure = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.756357] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.keyfile = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.756534] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.max_version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.756691] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.min_version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.756844] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.region_name = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.757009] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.retriable_status_codes = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.758857] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.service_name = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.758857] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.service_type = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.758857] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.split_loggers = False {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.758857] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.status_code_retries = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.758857] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.status_code_retry_delay = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.758857] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.timeout = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.758857] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.valid_interfaces = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.759178] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_limit.version = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.759178] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_reports.file_event_handler = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.759178] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.759178] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] oslo_reports.log_dir = None {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2826}} [ 403.759178] nova-conductor[54011]: DEBUG oslo_service.service [None req-ea1b7f72-8708-4635-ab6e-f629bf0a0eb4 None None] ******************************************************************************** {{(pid=54011) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2830}} [ 413.493411] nova-conductor[54011]: DEBUG dbcounter [-] [54011] Writing DB stats nova_api:SELECT=1 {{(pid=54011) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 413.586750] nova-conductor[54011]: DEBUG dbcounter [-] [54011] Writing DB stats nova_cell1:SELECT=1 {{(pid=54011) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 413.588988] nova-conductor[54011]: DEBUG dbcounter [-] [54011] Writing DB stats nova_cell0:SELECT=1 {{(pid=54011) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 418.670744] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=4,nova_cell1:INSERT=1,nova_cell1:UPDATE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 418.671088] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=4,nova_cell1:INSERT=1,nova_cell1:UPDATE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 488.143688] nova-conductor[54432]: DEBUG oslo_db.sqlalchemy.engines [None req-109569a9-6c1a-4dd8-97a0-a75aacdc1e0a None None] Parent process 54011 forked (54432) with an open database connection, which is being discarded and recreated. {{(pid=54432) checkout /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:438}} [ 488.145810] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writer thread running {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 498.148053] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 505.273070] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=7,nova_cell1:INSERT=1,nova_cell1:UPDATE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 510.792743] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=7,nova_cell1:INSERT=1,nova_cell1:UPDATE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 531.141375] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writer thread running {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 531.141629] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=2 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 531.902975] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Took 1.53 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 531.908894] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 531.934910] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 531.935457] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 531.940022] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 531.940022] nova-conductor[54432]: INFO dbcounter [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Registered counter for database nova_cell1 [ 531.943350] nova-conductor[54432]: DEBUG oslo_db.sqlalchemy.engines [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=54432) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 531.950750] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writer thread running {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 532.023436] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 532.023690] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 532.024130] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 532.024564] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 532.024751] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 532.024974] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 532.026276] nova-conductor[54432]: INFO dbcounter [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Registered counter for database nova_cell0 [ 532.033510] nova-conductor[54432]: DEBUG oslo_db.sqlalchemy.engines [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=54432) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 532.038507] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writer thread running {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 532.050141] nova-conductor[54432]: DEBUG nova.quota [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Getting quotas for project a182eb3e4cc24706883605441eac740d. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 532.053651] nova-conductor[54432]: DEBUG nova.quota [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Getting quotas for user fd02702ca4f740608114918e97833c16 and project a182eb3e4cc24706883605441eac740d. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 532.058905] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] [instance: ebd5745d-f073-429c-b609-a7520a539f2b] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 532.059485] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 532.059698] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 532.059966] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 532.066829] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] [instance: ebd5745d-f073-429c-b609-a7520a539f2b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 532.067566] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 532.067821] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 532.068045] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 532.102357] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 532.102584] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 532.102818] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 532.103094] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Acquiring lock "compute-rpcapi-router" {{(pid=54432) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 532.103242] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Acquired lock "compute-rpcapi-router" {{(pid=54432) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 532.103773] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-45ef31c6-92e0-4da4-9bae-6557b70ffd5a None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 532.104273] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-45ef31c6-92e0-4da4-9bae-6557b70ffd5a None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 532.104453] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-45ef31c6-92e0-4da4-9bae-6557b70ffd5a None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 532.104804] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-45ef31c6-92e0-4da4-9bae-6557b70ffd5a None None] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 532.104988] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-45ef31c6-92e0-4da4-9bae-6557b70ffd5a None None] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 532.105387] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-45ef31c6-92e0-4da4-9bae-6557b70ffd5a None None] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 532.111711] nova-conductor[54432]: INFO nova.compute.rpcapi [None req-45ef31c6-92e0-4da4-9bae-6557b70ffd5a None None] Automatically selected compute RPC version 6.3 from minimum service version 67 [ 532.112194] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-45ef31c6-92e0-4da4-9bae-6557b70ffd5a None None] Releasing lock "compute-rpcapi-router" {{(pid=54432) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 532.684216] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Took 1.54 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 532.737029] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 532.737205] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 532.739890] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 532.741203] nova-conductor[54431]: INFO dbcounter [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Registered counter for database nova_cell1 [ 532.749079] nova-conductor[54431]: DEBUG oslo_db.sqlalchemy.engines [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=54431) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 532.754891] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writer thread running {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 532.820295] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 532.820603] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 532.821186] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 532.821685] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 532.822097] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 532.822097] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 532.824500] nova-conductor[54431]: INFO dbcounter [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Registered counter for database nova_cell0 [ 532.836380] nova-conductor[54431]: DEBUG oslo_db.sqlalchemy.engines [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=54431) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} [ 532.842247] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writer thread running {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} [ 532.853298] nova-conductor[54431]: DEBUG nova.quota [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Getting quotas for project e4f68b27809f4bb58495a32dae15aa55. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 532.857295] nova-conductor[54431]: DEBUG nova.quota [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Getting quotas for user 0b7a1abad9e64791b4229f67d838f8b1 and project e4f68b27809f4bb58495a32dae15aa55. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 532.863129] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] [instance: 22ff04b4-85b7-4bf3-94c4-ba4dd323d6aa] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 532.863688] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 532.863896] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 532.864084] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 532.870100] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] [instance: 22ff04b4-85b7-4bf3-94c4-ba4dd323d6aa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 532.873020] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 532.873020] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 532.873020] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 532.902852] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 532.903125] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 532.903307] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 532.903582] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Acquiring lock "compute-rpcapi-router" {{(pid=54431) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 532.903878] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Acquired lock "compute-rpcapi-router" {{(pid=54431) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 532.904380] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc5da723-7764-4229-967d-a03b5bcf9260 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 532.904696] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc5da723-7764-4229-967d-a03b5bcf9260 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 532.904815] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc5da723-7764-4229-967d-a03b5bcf9260 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 532.905202] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc5da723-7764-4229-967d-a03b5bcf9260 None None] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 532.905401] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc5da723-7764-4229-967d-a03b5bcf9260 None None] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 532.905561] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc5da723-7764-4229-967d-a03b5bcf9260 None None] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 532.912553] nova-conductor[54431]: INFO nova.compute.rpcapi [None req-dc5da723-7764-4229-967d-a03b5bcf9260 None None] Automatically selected compute RPC version 6.3 from minimum service version 67 [ 532.913037] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc5da723-7764-4229-967d-a03b5bcf9260 None None] Releasing lock "compute-rpcapi-router" {{(pid=54431) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 534.395885] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 534.413210] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 534.413632] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 534.413851] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 534.457080] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 534.457521] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 534.457521] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 534.457826] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 534.458031] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 534.458201] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 534.467563] nova-conductor[54432]: DEBUG nova.quota [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Getting quotas for project a5a2a6c3b01640eba0a5b5612711385d. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 534.469842] nova-conductor[54432]: DEBUG nova.quota [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Getting quotas for user f05975529be14a03b5f165f134a770d4 and project a5a2a6c3b01640eba0a5b5612711385d. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 534.474876] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] [instance: 904c5d79-86f3-4d5b-b8ba-7e3d515b32c0] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 534.475805] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 534.476091] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 534.476242] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 534.490834] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] [instance: 904c5d79-86f3-4d5b-b8ba-7e3d515b32c0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 534.491581] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 534.491795] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 534.491964] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 534.511168] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 534.511323] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 534.512040] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 535.146056] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 535.161665] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 535.161891] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 535.162069] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 535.196290] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 535.196561] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 535.196737] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 535.197148] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 535.197338] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 535.197496] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 535.207632] nova-conductor[54431]: DEBUG nova.quota [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Getting quotas for project 27c9edfccbc6413ba8f969292695b2c7. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 535.210642] nova-conductor[54431]: DEBUG nova.quota [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Getting quotas for user 7c3697bf6ed14977aca6a0df8f5f4048 and project 27c9edfccbc6413ba8f969292695b2c7. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 535.216854] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] [instance: ddd212aa-9f4a-4b9d-aa60-cd4b64580f84] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 535.216854] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 535.216854] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 535.217207] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 535.224370] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] [instance: ddd212aa-9f4a-4b9d-aa60-cd4b64580f84] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 535.225509] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 535.225509] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 535.225509] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 535.243818] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 535.244173] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 535.244380] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fafae4bd-23d5-4104-be21-37060a3f4aa7 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 535.672115] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 535.693567] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 535.696028] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 535.696028] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 535.750502] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 535.750777] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 535.750915] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 535.751354] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 535.752261] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 535.752261] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 535.778758] nova-conductor[54432]: DEBUG nova.quota [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Getting quotas for project 26c92aca91614f018775d7e4949619d6. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 535.782967] nova-conductor[54432]: DEBUG nova.quota [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Getting quotas for user 7f4cf53af57b4bae9ab5788f2d663dc1 and project 26c92aca91614f018775d7e4949619d6. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 535.791031] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] [instance: 812c740f-eae7-4f6d-9e0d-8c16a4685a50] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 535.791587] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 535.791810] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 535.791979] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 535.802627] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] [instance: 812c740f-eae7-4f6d-9e0d-8c16a4685a50] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 535.803294] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 535.803529] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 535.803704] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 535.827440] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 535.827825] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 535.827895] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 536.543772] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 536.575152] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 536.575152] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 536.575340] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 536.619710] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 536.620233] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 536.620708] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 536.621281] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 536.621594] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 536.621871] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 536.632684] nova-conductor[54432]: DEBUG nova.quota [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Getting quotas for project 18249883469c4514a8300d99f32482a5. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 536.635637] nova-conductor[54432]: DEBUG nova.quota [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Getting quotas for user 11251536a5af4def955e95158d82fb81 and project 18249883469c4514a8300d99f32482a5. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 536.641970] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] [instance: 48fc0cbf-18a6-4bbc-8455-93800b911d6a] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 536.643406] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 536.643406] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 536.643406] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 536.648312] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] [instance: 48fc0cbf-18a6-4bbc-8455-93800b911d6a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 536.649536] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 536.649536] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 536.649831] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 536.673627] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 536.678108] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 536.678108] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 538.708032] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 538.724964] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 538.725549] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 538.725786] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 538.773019] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 538.773019] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 538.773019] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 538.773019] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 538.773232] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 538.773232] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 538.782986] nova-conductor[54432]: DEBUG nova.quota [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Getting quotas for project c3bcaeb2afd44f9a8621fbb43f65a1f7. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 538.786553] nova-conductor[54432]: DEBUG nova.quota [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Getting quotas for user bc4be38552f7413e9ed97d5bd407fdcd and project c3bcaeb2afd44f9a8621fbb43f65a1f7. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 538.793859] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] [instance: 99f77fa6-a625-4771-83c9-012f75ed1c7d] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 538.794390] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 538.794814] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 538.795695] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 538.800586] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] [instance: 99f77fa6-a625-4771-83c9-012f75ed1c7d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 538.801301] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 538.801507] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 538.801680] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 538.827725] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 538.827965] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 538.828193] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 545.202311] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell0:SELECT=3 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 545.228261] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=7,nova_cell1:SAVEPOINT=2,nova_cell1:INSERT=16,nova_cell1:RELEASE=2 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 545.239820] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=13,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 545.241777] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=10,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 547.847454] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 547.865570] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 547.868020] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 547.868020] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 547.912406] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 547.912625] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 547.913233] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 547.913233] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 547.913401] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 547.913526] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 547.926859] nova-conductor[54431]: DEBUG nova.quota [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Getting quotas for project 55d5905bb740438f87ef1bd846cbcb23. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 547.930571] nova-conductor[54431]: DEBUG nova.quota [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Getting quotas for user bfa56d5fb43340fe8b345221252368b7 and project 55d5905bb740438f87ef1bd846cbcb23. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 547.936567] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] [instance: 27103ce2-0962-4fce-9331-d74179b0510c] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 547.937265] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 547.937616] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 547.937959] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 547.943966] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] [instance: 27103ce2-0962-4fce-9331-d74179b0510c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 547.943966] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 547.944201] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 547.944374] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 547.972918] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 547.973160] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 547.973336] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ef1e9478-3440-44ab-9c16-bb88009b525b tempest-ServerDiagnosticsV248Test-1944406010 tempest-ServerDiagnosticsV248Test-1944406010-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 548.776906] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell0:SELECT=6 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 548.805263] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=17,nova_cell1:SAVEPOINT=5,nova_cell1:INSERT=41,nova_cell1:RELEASE=5 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 548.818180] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=33,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 548.825126] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=26,nova_api:UPDATE=3,nova_api:DELETE=4 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 550.113694] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 550.136813] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 550.136813] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 550.136813] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 550.182215] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 550.183651] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 550.183651] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 550.183651] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 550.183651] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 550.183789] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 550.196100] nova-conductor[54431]: DEBUG nova.quota [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Getting quotas for project 9512415a4f124e8c9120f6c115f78949. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 550.199505] nova-conductor[54431]: DEBUG nova.quota [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Getting quotas for user a4763a330d70462182e5bdccc2a08c25 and project 9512415a4f124e8c9120f6c115f78949. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 550.204638] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 2b7122b2-136d-4e0c-97eb-155b33905436] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 550.205346] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 550.205425] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 550.205627] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 550.211353] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 2b7122b2-136d-4e0c-97eb-155b33905436] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 550.212192] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 550.212284] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 550.212455] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 550.233166] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 550.233166] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 550.233166] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 551.427811] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 551.448590] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 551.448838] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 551.449024] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 551.483441] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 551.483677] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 551.483847] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 551.484245] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 551.484588] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 551.484588] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 551.493303] nova-conductor[54432]: DEBUG nova.quota [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Getting quotas for project b7d6c1f825564159823ed1b5e42e6b02. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 551.495391] nova-conductor[54432]: DEBUG nova.quota [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Getting quotas for user 3b351c419e664efd8066ab086962877d and project b7d6c1f825564159823ed1b5e42e6b02. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 551.500580] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] [instance: 76cf7f28-7413-4b06-bd30-ae22bbc2999e] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 551.501305] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 551.501516] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 551.501689] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 551.510697] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] [instance: 76cf7f28-7413-4b06-bd30-ae22bbc2999e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 551.511453] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 551.511673] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 551.511846] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 551.534546] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 551.534752] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 551.534933] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 552.706116] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 552.729545] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 552.730308] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 552.730308] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 552.816503] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 552.816503] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 552.816503] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 552.816503] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 552.817186] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 552.817186] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 552.828635] nova-conductor[54432]: DEBUG nova.quota [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Getting quotas for project ec3b357a819b4d7cbfd5fb7cfd6110c3. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 552.834010] nova-conductor[54432]: DEBUG nova.quota [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Getting quotas for user b50bcb588c294e339ed3bc90b34ec2cc and project ec3b357a819b4d7cbfd5fb7cfd6110c3. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 552.844067] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] [instance: 8d45497d-ba77-45b1-91f3-719b2689bee3] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 552.844067] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 552.845596] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 552.845596] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 552.852267] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] [instance: 8d45497d-ba77-45b1-91f3-719b2689bee3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 552.854145] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 552.855031] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 552.855031] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 552.875049] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 552.875338] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 552.875490] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 553.315887] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 553.342096] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 553.342324] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 553.342393] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 553.392039] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 553.392039] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 553.393144] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 553.393630] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 553.393908] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 553.394038] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 553.406030] nova-conductor[54432]: DEBUG nova.quota [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Getting quotas for project 5dbcb6d3c50043ebb6038807c1fe5c7e. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 553.409749] nova-conductor[54432]: DEBUG nova.quota [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Getting quotas for user d960e74fead14422b11cd3a2ee5df044 and project 5dbcb6d3c50043ebb6038807c1fe5c7e. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 553.418598] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] [instance: 7c5b9fab-bb90-4c70-b445-fe0b637dbd4a] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 553.419332] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 553.420428] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 553.420428] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 553.425297] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] [instance: 7c5b9fab-bb90-4c70-b445-fe0b637dbd4a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 553.426205] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 553.426205] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 553.426205] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 553.469710] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 553.485199] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 553.485199] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 553.485199] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 553.508023] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 553.508273] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 553.508703] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 553.544105] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 553.544105] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 553.544105] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 553.544281] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 553.544444] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 553.544610] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 553.556978] nova-conductor[54432]: DEBUG nova.quota [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Getting quotas for project 05837bc39d0040dbbcff7fe1236b5221. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 553.559638] nova-conductor[54432]: DEBUG nova.quota [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Getting quotas for user e8ac2ea6b5b2415e97542da5647a644a and project 05837bc39d0040dbbcff7fe1236b5221. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 553.570523] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] [instance: 3706243f-4768-4b44-9a5b-730bf2b18974] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 553.570766] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 553.571689] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 553.571689] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 553.579735] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] [instance: 3706243f-4768-4b44-9a5b-730bf2b18974] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 553.581256] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 553.581256] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 553.581256] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 553.607129] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 553.607129] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 553.607129] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 555.662545] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 555.691364] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 555.691609] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 555.691780] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 555.752204] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-fa8058c3-3d0b-413a-a23c-4052b47723d2 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Targeting cell 8fb5f300-ab58-4d7c-afe8-8d0b4be88dee(cell1) for conductor method rebuild_instance {{(pid=54432) wrapper /opt/stack/nova/nova/conductor/manager.py:95}} [ 555.754627] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-fa8058c3-3d0b-413a-a23c-4052b47723d2 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 555.754627] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-fa8058c3-3d0b-413a-a23c-4052b47723d2 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 555.754627] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-fa8058c3-3d0b-413a-a23c-4052b47723d2 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 555.767674] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 555.767892] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 555.768067] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 555.768475] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 555.768666] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 555.768873] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 555.777422] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-fa8058c3-3d0b-413a-a23c-4052b47723d2 tempest-ServersAdmin275Test-1946928354 tempest-ServersAdmin275Test-1946928354-project-member] [instance: ddd212aa-9f4a-4b9d-aa60-cd4b64580f84] No migration record for the rebuild/evacuate request. {{(pid=54432) rebuild_instance /opt/stack/nova/nova/conductor/manager.py:1227}} [ 555.793432] nova-conductor[54432]: DEBUG nova.quota [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Getting quotas for project e444950a637b4fdab2edb1d62aac4be5. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 555.796901] nova-conductor[54432]: DEBUG nova.quota [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Getting quotas for user c92b9d37f796479ca3b6d9f0ed1789c8 and project e444950a637b4fdab2edb1d62aac4be5. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 555.831072] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 9c7433a1-9d5b-4ef2-ad65-f1217f0e862c] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 555.831628] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 555.832639] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 555.832900] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 555.854020] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 9c7433a1-9d5b-4ef2-ad65-f1217f0e862c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 555.854020] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 555.854020] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 555.854277] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 555.896941] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 555.899421] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 555.899817] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 555.997819] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] [instance: ebd5745d-f073-429c-b609-a7520a539f2b] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port d19aae75-6444-4c54-9764-32b1260ecf19, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ebd5745d-f073-429c-b609-a7520a539f2b was re-scheduled: Binding failed for port d19aae75-6444-4c54-9764-32b1260ecf19, please check neutron logs for more information.\n'] [ 555.999370] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 555.999370] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ebd5745d-f073-429c-b609-a7520a539f2b.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ebd5745d-f073-429c-b609-a7520a539f2b. [ 555.999370] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] [instance: ebd5745d-f073-429c-b609-a7520a539f2b] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ebd5745d-f073-429c-b609-a7520a539f2b. [ 556.054710] nova-conductor[54431]: DEBUG nova.network.neutron [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] [instance: ebd5745d-f073-429c-b609-a7520a539f2b] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 556.716776] nova-conductor[54431]: DEBUG nova.network.neutron [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] [instance: ebd5745d-f073-429c-b609-a7520a539f2b] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 556.725433] nova-conductor[54431]: DEBUG nova.network.neutron [None req-e3eb74e0-6b45-4034-980f-c9629d2eb9b7 tempest-ServersAdminNegativeTestJSON-102187008 tempest-ServersAdminNegativeTestJSON-102187008-project-member] [instance: ebd5745d-f073-429c-b609-a7520a539f2b] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 557.841237] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] [instance: 22ff04b4-85b7-4bf3-94c4-ba4dd323d6aa] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 8fd4fe87-772a-4544-b201-6fed68efdb0a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 22ff04b4-85b7-4bf3-94c4-ba4dd323d6aa was re-scheduled: Binding failed for port 8fd4fe87-772a-4544-b201-6fed68efdb0a, please check neutron logs for more information.\n'] [ 557.842024] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 557.842336] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 22ff04b4-85b7-4bf3-94c4-ba4dd323d6aa.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 22ff04b4-85b7-4bf3-94c4-ba4dd323d6aa. [ 557.843030] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] [instance: 22ff04b4-85b7-4bf3-94c4-ba4dd323d6aa] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 22ff04b4-85b7-4bf3-94c4-ba4dd323d6aa. [ 557.872845] nova-conductor[54431]: DEBUG nova.network.neutron [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] [instance: 22ff04b4-85b7-4bf3-94c4-ba4dd323d6aa] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 557.928760] nova-conductor[54431]: DEBUG nova.network.neutron [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] [instance: 22ff04b4-85b7-4bf3-94c4-ba4dd323d6aa] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 557.936766] nova-conductor[54431]: DEBUG nova.network.neutron [None req-484bff5a-810f-4e34-ae21-8e618ef907c2 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954 tempest-FloatingIPsAssociationNegativeTestJSON-2127185954-project-member] [instance: 22ff04b4-85b7-4bf3-94c4-ba4dd323d6aa] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 557.954707] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 557.973076] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 557.973076] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 557.973076] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 558.008304] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 558.008397] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 558.008635] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 558.009030] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 558.009223] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 558.009415] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 558.026553] nova-conductor[54432]: DEBUG nova.quota [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Getting quotas for project 9c30565c40a845588906f604e8cb0e00. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 558.029363] nova-conductor[54432]: DEBUG nova.quota [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Getting quotas for user 9ad60a9ead734f08aa163c15c9a6b81d and project 9c30565c40a845588906f604e8cb0e00. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 558.035103] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] [instance: 0361a54c-f2ac-4f07-8cf5-4ecdc2c1f733] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 558.035585] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 558.036092] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 558.036092] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 558.041894] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] [instance: 0361a54c-f2ac-4f07-8cf5-4ecdc2c1f733] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 558.042527] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 558.042855] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 558.042974] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 558.068023] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 558.068074] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 558.068480] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 559.773671] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] [instance: 904c5d79-86f3-4d5b-b8ba-7e3d515b32c0] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port d253c0f7-c107-4caf-8d19-68e503db563f, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 904c5d79-86f3-4d5b-b8ba-7e3d515b32c0 was re-scheduled: Binding failed for port d253c0f7-c107-4caf-8d19-68e503db563f, please check neutron logs for more information.\n'] [ 559.774422] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 559.774653] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 904c5d79-86f3-4d5b-b8ba-7e3d515b32c0.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 904c5d79-86f3-4d5b-b8ba-7e3d515b32c0. [ 559.774885] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] [instance: 904c5d79-86f3-4d5b-b8ba-7e3d515b32c0] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 904c5d79-86f3-4d5b-b8ba-7e3d515b32c0. [ 559.837441] nova-conductor[54431]: DEBUG nova.network.neutron [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] [instance: 904c5d79-86f3-4d5b-b8ba-7e3d515b32c0] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 559.868523] nova-conductor[54431]: DEBUG nova.network.neutron [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] [instance: 904c5d79-86f3-4d5b-b8ba-7e3d515b32c0] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 559.870382] nova-conductor[54431]: DEBUG nova.network.neutron [None req-66269536-209c-4d6c-a3c0-6438dad90fe0 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] [instance: 904c5d79-86f3-4d5b-b8ba-7e3d515b32c0] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 560.123410] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 560.139742] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 560.139866] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 560.139939] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 560.188550] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell0:SELECT=2 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 560.192805] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 560.192909] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 560.193059] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 560.193442] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 560.193632] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 560.193837] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 560.208513] nova-conductor[54432]: DEBUG nova.quota [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Getting quotas for project a3c7f542e2214aeb9c76e9523151af1e. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 560.214692] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=6,nova_cell1:SAVEPOINT=2,nova_cell1:INSERT=16,nova_cell1:RELEASE=2 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 560.216763] nova-conductor[54432]: DEBUG nova.quota [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Getting quotas for user c07a7f3a5cce4d9a9ec16ec04c0cf198 and project a3c7f542e2214aeb9c76e9523151af1e. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 560.220326] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 560.229196] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] [instance: 7661165d-2731-456e-af25-da13c68fd9fb] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 560.229360] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=11,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 560.229402] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 560.229402] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 560.229402] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 560.235041] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] [instance: 7661165d-2731-456e-af25-da13c68fd9fb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 560.236107] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 560.236343] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 560.236518] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 560.263372] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 560.263427] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 560.263605] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 562.470903] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 562.489371] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 562.489620] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 562.489789] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 562.548141] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 562.548387] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 562.548591] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 562.549023] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 562.549756] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 562.549756] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 562.561602] nova-conductor[54431]: DEBUG nova.quota [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Getting quotas for project 7a5d82053bc44cb3a074fc125f58d068. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 562.564872] nova-conductor[54431]: DEBUG nova.quota [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Getting quotas for user a2ff9565717a4cceba792f68ffc03536 and project 7a5d82053bc44cb3a074fc125f58d068. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 562.571219] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] [instance: 037217df-a3ed-4cef-9478-a9cc576a2f7d] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 562.571353] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 562.572859] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 562.573087] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 562.584158] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] [instance: 037217df-a3ed-4cef-9478-a9cc576a2f7d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 562.584919] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 562.585812] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 562.585812] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 562.604460] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 562.604742] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 562.604886] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 566.761541] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 566.775992] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 566.776305] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 566.776489] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 566.813616] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 566.813849] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 566.814109] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 566.814468] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 566.814657] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 566.814820] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 566.825612] nova-conductor[54431]: DEBUG nova.quota [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Getting quotas for project ce8145b6c7084b12b5acacb131b54534. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 566.828659] nova-conductor[54431]: DEBUG nova.quota [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Getting quotas for user 6ddf09ee6bce4ab3b468d1ecd2095992 and project ce8145b6c7084b12b5acacb131b54534. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 566.834783] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] [instance: 9160ca3a-2aa2-4cfc-9900-3e4c770d2657] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 566.835333] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 566.835549] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 566.835721] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 566.843627] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] [instance: 9160ca3a-2aa2-4cfc-9900-3e4c770d2657] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 566.844409] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 566.844624] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 566.846071] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 566.860944] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 566.861189] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 566.861369] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 569.428599] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 569.454427] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 569.454427] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 569.454427] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 569.506315] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 569.506315] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 569.506315] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 569.506315] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 569.506315] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 569.506315] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 569.520806] nova-conductor[54431]: DEBUG nova.quota [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Getting quotas for project d6e3f664f5d3403289b586dcceb28424. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 569.530411] nova-conductor[54431]: DEBUG nova.quota [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Getting quotas for user f7f6adeea8054eefaf9d72e4264474c2 and project d6e3f664f5d3403289b586dcceb28424. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 569.543281] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] [instance: 81b1d279-dfad-4a2c-bcfb-043b13d77af6] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 569.543281] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 569.543281] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 569.543281] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 569.554133] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-912f518f-55c0-40c5-b2ec-8b2214614842 tempest-ServersAdmin275Test-1881070433 tempest-ServersAdmin275Test-1881070433-project-admin] Targeting cell 8fb5f300-ab58-4d7c-afe8-8d0b4be88dee(cell1) for conductor method rebuild_instance {{(pid=54431) wrapper /opt/stack/nova/nova/conductor/manager.py:95}} [ 569.554626] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-912f518f-55c0-40c5-b2ec-8b2214614842 tempest-ServersAdmin275Test-1881070433 tempest-ServersAdmin275Test-1881070433-project-admin] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 569.555044] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-912f518f-55c0-40c5-b2ec-8b2214614842 tempest-ServersAdmin275Test-1881070433 tempest-ServersAdmin275Test-1881070433-project-admin] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 569.555348] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-912f518f-55c0-40c5-b2ec-8b2214614842 tempest-ServersAdmin275Test-1881070433 tempest-ServersAdmin275Test-1881070433-project-admin] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 569.559022] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] [instance: 81b1d279-dfad-4a2c-bcfb-043b13d77af6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 569.559022] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 569.559022] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 569.559022] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 569.581511] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-912f518f-55c0-40c5-b2ec-8b2214614842 tempest-ServersAdmin275Test-1881070433 tempest-ServersAdmin275Test-1881070433-project-admin] [instance: ddd212aa-9f4a-4b9d-aa60-cd4b64580f84] No migration record for the rebuild/evacuate request. {{(pid=54431) rebuild_instance /opt/stack/nova/nova/conductor/manager.py:1227}} [ 569.604945] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 569.605424] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 569.605424] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 570.198510] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell0:SELECT=7 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 570.240051] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=25,nova_cell1:SAVEPOINT=7,nova_cell1:INSERT=58,nova_cell1:RELEASE=7,nova_cell1:UPDATE=3 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 570.252366] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=44,nova_api:DELETE=3,nova_api:UPDATE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 570.257626] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=42,nova_api:UPDATE=6,nova_api:DELETE=4 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 570.888911] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 570.906611] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 570.906919] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 570.907323] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 570.961387] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 570.961387] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 570.961387] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 570.961387] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 570.961387] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 570.961387] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 570.969147] nova-conductor[54432]: DEBUG nova.quota [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Getting quotas for project a5a2a6c3b01640eba0a5b5612711385d. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 570.972037] nova-conductor[54432]: DEBUG nova.quota [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Getting quotas for user f05975529be14a03b5f165f134a770d4 and project a5a2a6c3b01640eba0a5b5612711385d. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 570.978084] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] [instance: ccce765d-8df9-4837-b166-71e41bea6de7] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 570.978618] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 570.978839] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 570.979016] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 570.985841] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] [instance: ccce765d-8df9-4837-b166-71e41bea6de7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 570.986564] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 570.986746] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 570.986913] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 571.028999] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 571.029265] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 571.029434] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 571.227581] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] [instance: 812c740f-eae7-4f6d-9e0d-8c16a4685a50] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 900edb48-3a43-497c-9834-e3a23a708ba3, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 812c740f-eae7-4f6d-9e0d-8c16a4685a50 was re-scheduled: Binding failed for port 900edb48-3a43-497c-9834-e3a23a708ba3, please check neutron logs for more information.\n'] [ 571.230014] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 571.230014] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 812c740f-eae7-4f6d-9e0d-8c16a4685a50.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 812c740f-eae7-4f6d-9e0d-8c16a4685a50. [ 571.230014] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] [instance: 812c740f-eae7-4f6d-9e0d-8c16a4685a50] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 812c740f-eae7-4f6d-9e0d-8c16a4685a50. [ 571.292655] nova-conductor[54431]: DEBUG nova.network.neutron [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] [instance: 812c740f-eae7-4f6d-9e0d-8c16a4685a50] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 571.376140] nova-conductor[54431]: DEBUG nova.network.neutron [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] [instance: 812c740f-eae7-4f6d-9e0d-8c16a4685a50] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 571.387040] nova-conductor[54431]: DEBUG nova.network.neutron [None req-22f8f427-be8b-4f44-b491-2b7a8689330a tempest-ServerDiagnosticsTest-514470568 tempest-ServerDiagnosticsTest-514470568-project-member] [instance: 812c740f-eae7-4f6d-9e0d-8c16a4685a50] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 571.739284] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Took 0.51 seconds to select destinations for 2 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 571.762803] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 571.762803] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 571.762803] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 571.879531] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 571.879758] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 571.879929] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 571.921925] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 571.922155] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 571.922358] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 571.922822] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 571.922970] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 571.923139] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 571.947111] nova-conductor[54432]: DEBUG nova.quota [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Getting quotas for project 7fa50ab2c6ba4659afee1f146822a981. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 571.953839] nova-conductor[54432]: DEBUG nova.quota [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Getting quotas for user d896dc782fdf4c13b943c0336d255452 and project 7fa50ab2c6ba4659afee1f146822a981. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 571.964084] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 287c3404-9fc1-4369-b9a4-17da918bf78d] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 571.964137] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 571.966337] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 571.966337] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 571.972274] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 287c3404-9fc1-4369-b9a4-17da918bf78d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 571.972638] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 571.973245] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 571.974018] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 571.992444] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 571.992796] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 571.992985] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 572.000377] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 6680c707-aa59-4e00-a2b1-38d465cc6497] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 572.000992] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 572.001137] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 572.001317] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 572.008571] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 6680c707-aa59-4e00-a2b1-38d465cc6497] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 572.009235] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 572.009235] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 572.009408] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 572.027450] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 572.028591] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 572.028591] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 575.418210] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=277,nova_cell1:UPDATE=122,nova_cell1:INSERT=18,nova_cell1:SAVEPOINT=21,nova_cell1:RELEASE=21 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 575.476065] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 575.495975] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 575.496448] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 575.497018] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 575.546384] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 575.546817] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 575.547137] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 575.547756] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 575.549083] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 575.549083] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 575.563261] nova-conductor[54432]: DEBUG nova.quota [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Getting quotas for project 2e3c5bed90a143e2baa34a7df2895fa5. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 575.567450] nova-conductor[54432]: DEBUG nova.quota [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Getting quotas for user 34030955a6164c8f8f90b42bbf9c5551 and project 2e3c5bed90a143e2baa34a7df2895fa5. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 575.579517] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] [instance: 2a963e2c-7ee3-41a4-842b-06c151d27d84] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 575.582498] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 575.582498] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 575.582498] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 575.586488] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] [instance: 2a963e2c-7ee3-41a4-842b-06c151d27d84] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 575.587248] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 575.587468] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 575.587641] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 575.613986] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 575.614261] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 575.614413] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 577.994592] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 578.016971] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 578.019108] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 578.019325] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 578.062689] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 578.062689] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 578.062689] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 578.062958] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 578.063164] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 578.063330] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 578.083355] nova-conductor[54431]: DEBUG nova.quota [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Getting quotas for project 3ebc1a5cf86945f381bc1942d2f83754. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 578.085624] nova-conductor[54431]: DEBUG nova.quota [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Getting quotas for user c97a57aba8534d8691106d790547d0d1 and project 3ebc1a5cf86945f381bc1942d2f83754. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 578.096247] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] [instance: 5e18d3cc-62bf-4f12-8e89-2e1bcad15d49] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 578.096704] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 578.098384] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 578.098384] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 578.102808] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] [instance: 5e18d3cc-62bf-4f12-8e89-2e1bcad15d49] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 578.103650] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 578.103650] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 578.103650] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 578.131426] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 578.131426] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 578.131426] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 579.673592] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] [instance: 48fc0cbf-18a6-4bbc-8455-93800b911d6a] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port eff92d51-a578-421d-98e9-d5ffbfb11bb1, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 48fc0cbf-18a6-4bbc-8455-93800b911d6a was re-scheduled: Binding failed for port eff92d51-a578-421d-98e9-d5ffbfb11bb1, please check neutron logs for more information.\n'] [ 579.674379] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 579.674573] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 48fc0cbf-18a6-4bbc-8455-93800b911d6a.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 48fc0cbf-18a6-4bbc-8455-93800b911d6a. [ 579.675440] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] [instance: 48fc0cbf-18a6-4bbc-8455-93800b911d6a] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 48fc0cbf-18a6-4bbc-8455-93800b911d6a. [ 579.717315] nova-conductor[54431]: DEBUG nova.network.neutron [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] [instance: 48fc0cbf-18a6-4bbc-8455-93800b911d6a] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 579.762222] nova-conductor[54431]: DEBUG nova.network.neutron [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] [instance: 48fc0cbf-18a6-4bbc-8455-93800b911d6a] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 579.770392] nova-conductor[54431]: DEBUG nova.network.neutron [None req-01135f88-abb2-4a89-84bb-7d712af48c5b tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] [instance: 48fc0cbf-18a6-4bbc-8455-93800b911d6a] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 581.027908] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=298,nova_cell1:UPDATE=129,nova_cell1:INSERT=20,nova_cell1:SAVEPOINT=17,nova_cell1:RELEASE=17 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 583.397751] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 583.414878] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 583.415132] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 583.415312] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 583.470414] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 583.470650] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 583.470833] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 583.471235] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 583.471464] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 583.471574] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 583.508067] nova-conductor[54432]: DEBUG nova.quota [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Getting quotas for project 96b5ee8496a84fb09aaaeef5d6f8a5c7. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 583.518636] nova-conductor[54432]: DEBUG nova.quota [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Getting quotas for user c9f0e85cdee84517959179c0a693e265 and project 96b5ee8496a84fb09aaaeef5d6f8a5c7. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 583.531272] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] [instance: 0464da14-cb61-4f8f-9bd7-16f970aec0e7] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 583.534032] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 583.534032] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 583.534032] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 583.549527] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] [instance: 0464da14-cb61-4f8f-9bd7-16f970aec0e7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 583.549527] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 583.549527] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 583.549527] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 583.570649] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 583.570916] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 583.571149] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 584.336705] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] [instance: 99f77fa6-a625-4771-83c9-012f75ed1c7d] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port f71924e4-d6df-4e41-b251-783bd1c767a7, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 99f77fa6-a625-4771-83c9-012f75ed1c7d was re-scheduled: Binding failed for port f71924e4-d6df-4e41-b251-783bd1c767a7, please check neutron logs for more information.\n'] [ 584.336705] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 584.336705] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 99f77fa6-a625-4771-83c9-012f75ed1c7d.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 99f77fa6-a625-4771-83c9-012f75ed1c7d. [ 584.336705] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] [instance: 99f77fa6-a625-4771-83c9-012f75ed1c7d] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 99f77fa6-a625-4771-83c9-012f75ed1c7d. [ 584.385636] nova-conductor[54432]: DEBUG nova.network.neutron [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] [instance: 99f77fa6-a625-4771-83c9-012f75ed1c7d] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 584.488585] nova-conductor[54432]: DEBUG nova.network.neutron [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] [instance: 99f77fa6-a625-4771-83c9-012f75ed1c7d] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 584.510212] nova-conductor[54432]: DEBUG nova.network.neutron [None req-674d8a4c-11a0-46bb-90a5-40d4f14ed82c tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] [instance: 99f77fa6-a625-4771-83c9-012f75ed1c7d] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 587.925033] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 587.925281] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 587.925451] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 588.070264] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell0:SELECT=4 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 588.108663] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=16,nova_cell1:SAVEPOINT=4,nova_cell1:INSERT=33,nova_cell1:RELEASE=4,nova_cell1:UPDATE=3 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 588.120583] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=24,nova_api:UPDATE=2,nova_api:DELETE=3 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 588.126117] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=26,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 588.469978] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 588.488335] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.488747] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.488845] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 588.544135] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.544135] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.544135] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 588.544135] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.544135] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.544593] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 588.556782] nova-conductor[54432]: DEBUG nova.quota [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Getting quotas for project 9e8b3e6c662f4075b818d1a01daebd5d. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 588.562164] nova-conductor[54432]: DEBUG nova.quota [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Getting quotas for user c80d636b389e4e1286595024e5f98d8e and project 9e8b3e6c662f4075b818d1a01daebd5d. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 588.570400] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] [instance: c1cd40be-a66c-4ed9-b329-6d94509b19cf] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 588.572425] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.572425] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.572425] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 588.578678] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] [instance: c1cd40be-a66c-4ed9-b329-6d94509b19cf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 588.582336] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.582336] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.582428] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 588.604098] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 588.604098] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 588.604098] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 589.653303] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 589.672020] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 589.672664] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 589.672664] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 589.779230] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 589.779498] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 589.779688] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 589.780106] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 589.780396] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 589.780499] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 589.804663] nova-conductor[54432]: DEBUG nova.quota [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Getting quotas for project 18249883469c4514a8300d99f32482a5. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 589.807839] nova-conductor[54432]: DEBUG nova.quota [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Getting quotas for user 11251536a5af4def955e95158d82fb81 and project 18249883469c4514a8300d99f32482a5. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 589.815034] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] [instance: e654ab80-4443-4f60-9d51-6730e1907ffe] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 589.817130] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 589.817887] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 589.817887] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 589.825699] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] [instance: e654ab80-4443-4f60-9d51-6730e1907ffe] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 589.826713] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 589.826860] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 589.827067] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 589.865363] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 589.865363] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 589.865363] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 590.991895] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 591.010311] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 591.010579] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 591.010757] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 591.069057] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 591.069057] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 591.069057] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 591.069310] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 591.069501] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 591.069658] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 591.083081] nova-conductor[54431]: DEBUG nova.quota [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Getting quotas for project 9334d955af1f4bbf815935fdc0ff3edf. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 591.087746] nova-conductor[54431]: DEBUG nova.quota [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Getting quotas for user 5b24cb55df0941d88665d71e23a27820 and project 9334d955af1f4bbf815935fdc0ff3edf. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 591.098404] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] [instance: 28832fde-2434-4d5a-ae62-550b8c0eb829] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 591.098878] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 591.100900] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 591.100900] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 591.109469] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] [instance: 28832fde-2434-4d5a-ae62-550b8c0eb829] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 591.109469] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 591.109469] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 591.109469] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 591.127354] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 591.127599] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 591.127771] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 595.327380] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 595.340971] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 595.341251] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 595.341438] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 595.377458] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 595.377920] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 595.378152] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 595.378534] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 595.378727] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 595.378902] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 595.388411] nova-conductor[54432]: DEBUG nova.quota [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Getting quotas for project b1e75334f9c84afb9adbcdd22ce9834a. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 595.391640] nova-conductor[54432]: DEBUG nova.quota [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Getting quotas for user d0057e858a1e430e9fcb0f49da3935b3 and project b1e75334f9c84afb9adbcdd22ce9834a. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 595.401083] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] [instance: 873731fb-8d97-41e7-830b-b9ae4281a737] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 595.401629] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 595.401892] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 595.402247] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 595.408118] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] [instance: 873731fb-8d97-41e7-830b-b9ae4281a737] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 595.408782] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 595.408993] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 595.409174] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 595.431410] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 595.431703] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 595.431883] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 597.072207] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 2b7122b2-136d-4e0c-97eb-155b33905436] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 42bcc024-8460-4983-9bc4-d3545aa63650, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 2b7122b2-136d-4e0c-97eb-155b33905436 was re-scheduled: Binding failed for port 42bcc024-8460-4983-9bc4-d3545aa63650, please check neutron logs for more information.\n'] [ 597.073856] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 597.074166] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2b7122b2-136d-4e0c-97eb-155b33905436.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2b7122b2-136d-4e0c-97eb-155b33905436. [ 597.074470] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 2b7122b2-136d-4e0c-97eb-155b33905436] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2b7122b2-136d-4e0c-97eb-155b33905436. [ 597.116497] nova-conductor[54432]: DEBUG nova.network.neutron [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 2b7122b2-136d-4e0c-97eb-155b33905436] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 597.167643] nova-conductor[54432]: DEBUG nova.network.neutron [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 2b7122b2-136d-4e0c-97eb-155b33905436] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 597.172831] nova-conductor[54432]: DEBUG nova.network.neutron [None req-01bcd023-3223-4eff-b64d-5aa8ff88166a tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 2b7122b2-136d-4e0c-97eb-155b33905436] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 598.414607] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 598.433119] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.433119] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.433119] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.466897] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.466897] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.466897] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.466897] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.467174] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.467425] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.480621] nova-conductor[54432]: DEBUG nova.quota [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Getting quotas for project c3bcaeb2afd44f9a8621fbb43f65a1f7. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 598.483977] nova-conductor[54432]: DEBUG nova.quota [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Getting quotas for user bc4be38552f7413e9ed97d5bd407fdcd and project c3bcaeb2afd44f9a8621fbb43f65a1f7. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 598.490216] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] [instance: 9ca13604-2b1b-4d47-8a23-8137384a87a6] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 598.490489] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.490685] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.490845] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.495697] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] [instance: 9ca13604-2b1b-4d47-8a23-8137384a87a6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 598.496313] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.499972] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.499972] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.519126] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.519439] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.519528] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.725400] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 598.744863] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.745146] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.747038] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.807649] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.808171] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.808171] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.808860] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.808951] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.809082] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.827624] nova-conductor[54432]: DEBUG nova.quota [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Getting quotas for project 9ca4a4fc017a45cfbd044295ee6864bf. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 598.840039] nova-conductor[54432]: DEBUG nova.quota [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Getting quotas for user c790f5999395473dadd46868c37897d5 and project 9ca4a4fc017a45cfbd044295ee6864bf. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 598.861879] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] [instance: 3b5fb42c-8a81-4575-ba8f-424a101d11ae] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 598.864907] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.866725] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.866929] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.879020] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] [instance: 3b5fb42c-8a81-4575-ba8f-424a101d11ae] block_device_mapping [BlockDeviceMapping(attachment_id=aee7de93-3e41-4c78-a15a-72299bf32a2a,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='7a0a7388-8cdd-491f-9a66-930875cdefd6',volume_size=1,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 598.880397] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.880397] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.880397] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 598.914060] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 598.914060] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 598.914060] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 600.372098] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 600.392996] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 600.392996] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 600.392996] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 600.505567] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 600.505819] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 600.506139] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 600.506563] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 600.506818] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 600.507103] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 600.540522] nova-conductor[54431]: DEBUG nova.quota [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Getting quotas for project 2560c0a335224866815786b4185bb761. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 600.545984] nova-conductor[54431]: DEBUG nova.quota [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Getting quotas for user 8d900d87cd0b4d3f83ca573311a26304 and project 2560c0a335224866815786b4185bb761. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 600.556953] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] [instance: 0899249c-880c-48de-b2c3-2c407115463e] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 600.557355] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 600.557563] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 600.557728] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 600.562192] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 600.564722] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] [instance: 0899249c-880c-48de-b2c3-2c407115463e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 600.565694] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 600.565694] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 600.566000] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 600.582919] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 600.583427] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 600.583650] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 600.602691] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 600.602988] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 600.603241] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 600.635683] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 600.636013] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 600.636111] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 600.636622] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 600.636680] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 600.636860] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 600.646115] nova-conductor[54431]: DEBUG nova.quota [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Getting quotas for project 67dfc1220d0e4a118bf966b471761b74. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 600.649828] nova-conductor[54431]: DEBUG nova.quota [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Getting quotas for user 8a58d14b9cc84049a40b16654df2cbdd and project 67dfc1220d0e4a118bf966b471761b74. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 600.655902] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] [instance: a4a9c3d1-70f5-4521-9325-e3a7f6e7de1b] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 600.656446] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 600.656661] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 600.656830] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 600.660197] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] [instance: a4a9c3d1-70f5-4521-9325-e3a7f6e7de1b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 600.660859] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 600.661076] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 600.661255] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 600.678723] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 600.678974] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 600.679193] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 601.564080] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] [instance: 76cf7f28-7413-4b06-bd30-ae22bbc2999e] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 60f230d6-abe8-45a2-896b-081db4709502, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 76cf7f28-7413-4b06-bd30-ae22bbc2999e was re-scheduled: Binding failed for port 60f230d6-abe8-45a2-896b-081db4709502, please check neutron logs for more information.\n'] [ 601.565261] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 601.565578] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 76cf7f28-7413-4b06-bd30-ae22bbc2999e.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 76cf7f28-7413-4b06-bd30-ae22bbc2999e. [ 601.565984] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] [instance: 76cf7f28-7413-4b06-bd30-ae22bbc2999e] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 76cf7f28-7413-4b06-bd30-ae22bbc2999e. [ 601.606446] nova-conductor[54432]: DEBUG nova.network.neutron [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] [instance: 76cf7f28-7413-4b06-bd30-ae22bbc2999e] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 601.649312] nova-conductor[54432]: DEBUG nova.network.neutron [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] [instance: 76cf7f28-7413-4b06-bd30-ae22bbc2999e] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 601.653796] nova-conductor[54432]: DEBUG nova.network.neutron [None req-2605a6f9-bb56-4a61-8489-4216b170e66b tempest-ServerExternalEventsTest-941014284 tempest-ServerExternalEventsTest-941014284-project-member] [instance: 76cf7f28-7413-4b06-bd30-ae22bbc2999e] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 602.832805] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 602.850898] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 602.851165] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 602.851342] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 602.898835] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 602.899168] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 602.899306] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 602.899868] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 602.900478] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 602.900657] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 602.918567] nova-conductor[54432]: DEBUG nova.quota [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Getting quotas for project 5d94d12e6597456dbbce7377306815cf. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 602.927306] nova-conductor[54432]: DEBUG nova.quota [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Getting quotas for user c94d4520a1b94af6b4c8093e4951fd4f and project 5d94d12e6597456dbbce7377306815cf. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 602.935757] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] [instance: da2b873c-0cac-4a7d-8639-b13e0f31b7b6] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 602.936292] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 602.936558] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 602.936732] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 602.944653] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] [instance: da2b873c-0cac-4a7d-8639-b13e0f31b7b6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 602.946409] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 602.946409] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 602.946409] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 602.970499] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 602.970499] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 602.970499] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 603.559509] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 603.576887] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 603.576887] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 603.576887] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 603.614020] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 603.614020] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 603.614020] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 603.614020] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 603.614020] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 603.614020] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 603.628643] nova-conductor[54432]: DEBUG nova.quota [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Getting quotas for project 9512415a4f124e8c9120f6c115f78949. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 603.632740] nova-conductor[54432]: DEBUG nova.quota [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Getting quotas for user a4763a330d70462182e5bdccc2a08c25 and project 9512415a4f124e8c9120f6c115f78949. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 603.638621] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 3d9ac347-ffcc-4840-8090-811dbd2865ef] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 603.640236] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 603.640236] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 603.640236] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 603.647872] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 3d9ac347-ffcc-4840-8090-811dbd2865ef] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 603.647872] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 603.647872] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 603.647872] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 603.664690] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 603.664917] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 603.665099] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 606.558425] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 606.573550] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 606.573769] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 606.574686] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 606.616047] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 606.616306] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 606.616477] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 606.616901] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 606.617038] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 606.617201] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 606.625186] nova-conductor[54432]: DEBUG nova.quota [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Getting quotas for project 96ee79ce9ee544ebb82b5562d682e849. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 606.627316] nova-conductor[54432]: DEBUG nova.quota [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Getting quotas for user 7f85a963ad4543a2a0cc460b355a9d26 and project 96ee79ce9ee544ebb82b5562d682e849. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 606.632485] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] [instance: 3649431e-f856-4a28-9af7-8f101c3fc2d6] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 606.633606] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 606.633606] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 606.633606] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 606.638151] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] [instance: 3649431e-f856-4a28-9af7-8f101c3fc2d6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 606.638854] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 606.639219] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 606.639294] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 606.658500] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 606.658717] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 606.658887] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 607.217082] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] [instance: 8d45497d-ba77-45b1-91f3-719b2689bee3] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 149941b7-e36f-4036-b7de-c89202397eac, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 8d45497d-ba77-45b1-91f3-719b2689bee3 was re-scheduled: Binding failed for port 149941b7-e36f-4036-b7de-c89202397eac, please check neutron logs for more information.\n'] [ 607.217701] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 607.217983] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8d45497d-ba77-45b1-91f3-719b2689bee3.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8d45497d-ba77-45b1-91f3-719b2689bee3. [ 607.220014] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] [instance: 8d45497d-ba77-45b1-91f3-719b2689bee3] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8d45497d-ba77-45b1-91f3-719b2689bee3. [ 607.255654] nova-conductor[54432]: DEBUG nova.network.neutron [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] [instance: 8d45497d-ba77-45b1-91f3-719b2689bee3] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 607.288609] nova-conductor[54432]: DEBUG nova.network.neutron [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] [instance: 8d45497d-ba77-45b1-91f3-719b2689bee3] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 607.292842] nova-conductor[54432]: DEBUG nova.network.neutron [None req-c0934103-e766-4f78-851e-90f324c75734 tempest-ServersTestJSON-162904896 tempest-ServersTestJSON-162904896-project-member] [instance: 8d45497d-ba77-45b1-91f3-719b2689bee3] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 609.045621] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 609.081533] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.081878] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.082197] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 609.129992] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.130251] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.130420] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 609.130799] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.130979] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.131860] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 609.143053] nova-conductor[54431]: DEBUG nova.quota [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Getting quotas for project 3f11318c300c4c308f3f8a14db355954. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 609.145868] nova-conductor[54431]: DEBUG nova.quota [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Getting quotas for user 4762545dc5e14c16b5cbb1bad27d90a2 and project 3f11318c300c4c308f3f8a14db355954. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 609.153437] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 71fa08bc-a5c7-459c-b65b-5dacd9537281] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 609.153437] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.153437] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.153801] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 609.157352] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 71fa08bc-a5c7-459c-b65b-5dacd9537281] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 609.158852] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.158852] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.158983] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 609.189783] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.189783] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.189783] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 609.560249] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 609.579664] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.579903] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.580243] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 609.624046] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.624298] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.624477] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 609.624924] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.625046] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.625686] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 609.636235] nova-conductor[54432]: DEBUG nova.quota [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Getting quotas for project e176d3260a434fa2b4d2a113789c970a. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 609.641524] nova-conductor[54432]: DEBUG nova.quota [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Getting quotas for user 154e9fc2b68e4dc299ff4ff2419c9d33 and project e176d3260a434fa2b4d2a113789c970a. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 609.647154] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] [instance: 6ae11b59-68af-482a-ad65-543a56d7aaa4] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 609.647586] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.647795] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.647964] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 609.656786] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] [instance: 6ae11b59-68af-482a-ad65-543a56d7aaa4] block_device_mapping [BlockDeviceMapping(attachment_id=0f38a77b-565c-4cd3-aac4-38a63267a8a1,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='ad811913-0e4c-42df-8394-485e010a5c02',volume_size=1,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 609.657491] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.657711] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.657912] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 609.677063] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 609.677320] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 609.677495] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 615.994665] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] [instance: 7c5b9fab-bb90-4c70-b445-fe0b637dbd4a] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 1bf2c3a4-bfd7-4a69-a486-3f07a007056e, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 7c5b9fab-bb90-4c70-b445-fe0b637dbd4a was re-scheduled: Binding failed for port 1bf2c3a4-bfd7-4a69-a486-3f07a007056e, please check neutron logs for more information.\n'] [ 615.994665] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 615.994665] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7c5b9fab-bb90-4c70-b445-fe0b637dbd4a.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7c5b9fab-bb90-4c70-b445-fe0b637dbd4a. [ 615.994665] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] [instance: 7c5b9fab-bb90-4c70-b445-fe0b637dbd4a] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7c5b9fab-bb90-4c70-b445-fe0b637dbd4a. [ 616.029129] nova-conductor[54432]: DEBUG nova.network.neutron [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] [instance: 7c5b9fab-bb90-4c70-b445-fe0b637dbd4a] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 616.045222] nova-conductor[54432]: DEBUG nova.network.neutron [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] [instance: 7c5b9fab-bb90-4c70-b445-fe0b637dbd4a] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 616.048186] nova-conductor[54432]: DEBUG nova.network.neutron [None req-d0a08764-f186-403c-9a54-545a14e71922 tempest-ServersWithSpecificFlavorTestJSON-240076549 tempest-ServersWithSpecificFlavorTestJSON-240076549-project-member] [instance: 7c5b9fab-bb90-4c70-b445-fe0b637dbd4a] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 616.598040] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 616.614630] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 616.614860] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 616.615295] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 616.656206] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 616.656491] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 616.656709] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 616.657231] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 616.657435] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 616.657600] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 616.671241] nova-conductor[54431]: DEBUG nova.quota [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Getting quotas for project 5fe57bb058af414d9a696248dd14fa7b. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 616.674198] nova-conductor[54431]: DEBUG nova.quota [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Getting quotas for user 5a4553a28ba342a98948136b4d24874a and project 5fe57bb058af414d9a696248dd14fa7b. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 616.682401] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] [instance: 6d8b7855-567c-44c3-90ee-fc0daec87f46] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 616.682706] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 616.682919] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 616.683109] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 616.688065] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] [instance: 6d8b7855-567c-44c3-90ee-fc0daec87f46] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 616.688782] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 616.688992] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 616.689181] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 616.715040] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 616.715301] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 616.715479] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 617.850042] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] [instance: 3706243f-4768-4b44-9a5b-730bf2b18974] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 3ecff7db-5412-4e0b-bc53-9a1924aa0f90, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 3706243f-4768-4b44-9a5b-730bf2b18974 was re-scheduled: Binding failed for port 3ecff7db-5412-4e0b-bc53-9a1924aa0f90, please check neutron logs for more information.\n'] [ 617.850463] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 617.850749] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3706243f-4768-4b44-9a5b-730bf2b18974.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3706243f-4768-4b44-9a5b-730bf2b18974. [ 617.850973] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] [instance: 3706243f-4768-4b44-9a5b-730bf2b18974] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3706243f-4768-4b44-9a5b-730bf2b18974. [ 617.888767] nova-conductor[54432]: DEBUG nova.network.neutron [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] [instance: 3706243f-4768-4b44-9a5b-730bf2b18974] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 617.915193] nova-conductor[54432]: DEBUG nova.network.neutron [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] [instance: 3706243f-4768-4b44-9a5b-730bf2b18974] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 617.918855] nova-conductor[54432]: DEBUG nova.network.neutron [None req-36db2140-89f2-4fd4-bb0d-71db22b3a4be tempest-VolumesAssistedSnapshotsTest-1771293483 tempest-VolumesAssistedSnapshotsTest-1771293483-project-member] [instance: 3706243f-4768-4b44-9a5b-730bf2b18974] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 619.115984] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 619.134022] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 619.134022] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 619.134022] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 619.209771] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 619.209771] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 619.209771] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 619.209771] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 619.209771] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 619.209771] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 619.219912] nova-conductor[54431]: DEBUG nova.quota [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Getting quotas for project 5bb308bb00f048f4bd47b3cd77843a58. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 619.223969] nova-conductor[54431]: DEBUG nova.quota [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Getting quotas for user 70fa403562d44a82b91063fd0b4a2664 and project 5bb308bb00f048f4bd47b3cd77843a58. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 619.233960] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] [instance: 8a769120-906c-4e7c-ac19-b7895039efc1] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 619.233960] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 619.233960] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 619.233960] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 619.243889] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] [instance: 8a769120-906c-4e7c-ac19-b7895039efc1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 619.244665] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 619.244792] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 619.244965] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 619.265328] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 619.265553] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 619.265722] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 619.629524] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell0:SELECT=13 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 619.661963] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=41,nova_cell1:INSERT=109,nova_cell1:SAVEPOINT=9,nova_cell1:RELEASE=9 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 619.668884] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=86,nova_api:UPDATE=10,nova_api:DELETE=6 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 619.672846] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=75,nova_api:DELETE=8,nova_api:UPDATE=4 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 619.870684] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 9c7433a1-9d5b-4ef2-ad65-f1217f0e862c] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b5877af8-44ec-41fc-bf83-41d3e7cf35f6, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 9c7433a1-9d5b-4ef2-ad65-f1217f0e862c was re-scheduled: Binding failed for port b5877af8-44ec-41fc-bf83-41d3e7cf35f6, please check neutron logs for more information.\n'] [ 619.871334] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 619.871581] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9c7433a1-9d5b-4ef2-ad65-f1217f0e862c.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9c7433a1-9d5b-4ef2-ad65-f1217f0e862c. [ 619.872688] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 9c7433a1-9d5b-4ef2-ad65-f1217f0e862c] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9c7433a1-9d5b-4ef2-ad65-f1217f0e862c. [ 619.910881] nova-conductor[54431]: DEBUG nova.network.neutron [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 9c7433a1-9d5b-4ef2-ad65-f1217f0e862c] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 619.963846] nova-conductor[54431]: DEBUG nova.network.neutron [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 9c7433a1-9d5b-4ef2-ad65-f1217f0e862c] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 619.970142] nova-conductor[54431]: DEBUG nova.network.neutron [None req-03d5a889-1d0a-4746-9844-58ca11408bd2 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 9c7433a1-9d5b-4ef2-ad65-f1217f0e862c] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 622.712453] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 622.734466] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 622.734632] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 622.734750] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 622.782018] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 622.782018] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 622.782116] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 622.782548] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 622.782727] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 622.782856] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 622.795125] nova-conductor[54432]: DEBUG nova.quota [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Getting quotas for project e444950a637b4fdab2edb1d62aac4be5. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 622.798281] nova-conductor[54432]: DEBUG nova.quota [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Getting quotas for user c92b9d37f796479ca3b6d9f0ed1789c8 and project e444950a637b4fdab2edb1d62aac4be5. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 622.805676] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 75b04055-8c7e-4659-9451-ddcd6d39fe2e] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 622.805676] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 622.805676] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 622.805676] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 622.810197] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 75b04055-8c7e-4659-9451-ddcd6d39fe2e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 622.810876] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 622.811113] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 622.811286] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 622.828481] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 622.828814] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 622.828897] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 624.785322] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] [instance: 0361a54c-f2ac-4f07-8cf5-4ecdc2c1f733] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 10bc8270-f0af-4ac1-984d-dedf660a797d, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 0361a54c-f2ac-4f07-8cf5-4ecdc2c1f733 was re-scheduled: Binding failed for port 10bc8270-f0af-4ac1-984d-dedf660a797d, please check neutron logs for more information.\n'] [ 624.786020] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 624.786301] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0361a54c-f2ac-4f07-8cf5-4ecdc2c1f733.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0361a54c-f2ac-4f07-8cf5-4ecdc2c1f733. [ 624.786754] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] [instance: 0361a54c-f2ac-4f07-8cf5-4ecdc2c1f733] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0361a54c-f2ac-4f07-8cf5-4ecdc2c1f733. [ 624.817853] nova-conductor[54432]: DEBUG nova.network.neutron [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] [instance: 0361a54c-f2ac-4f07-8cf5-4ecdc2c1f733] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 624.833163] nova-conductor[54432]: DEBUG nova.network.neutron [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] [instance: 0361a54c-f2ac-4f07-8cf5-4ecdc2c1f733] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 624.839062] nova-conductor[54432]: DEBUG nova.network.neutron [None req-0468becc-c67e-4f75-9c76-8f47c0479491 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] [instance: 0361a54c-f2ac-4f07-8cf5-4ecdc2c1f733] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 627.009105] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] [instance: 7661165d-2731-456e-af25-da13c68fd9fb] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 6eb83bc1-e5fd-4aba-bb63-fc8d979dbabf, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 7661165d-2731-456e-af25-da13c68fd9fb was re-scheduled: Binding failed for port 6eb83bc1-e5fd-4aba-bb63-fc8d979dbabf, please check neutron logs for more information.\n'] [ 627.011207] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 627.011207] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7661165d-2731-456e-af25-da13c68fd9fb.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7661165d-2731-456e-af25-da13c68fd9fb. [ 627.011207] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] [instance: 7661165d-2731-456e-af25-da13c68fd9fb] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7661165d-2731-456e-af25-da13c68fd9fb. [ 627.060359] nova-conductor[54431]: DEBUG nova.network.neutron [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] [instance: 7661165d-2731-456e-af25-da13c68fd9fb] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 627.099416] nova-conductor[54431]: DEBUG nova.network.neutron [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] [instance: 7661165d-2731-456e-af25-da13c68fd9fb] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 627.105417] nova-conductor[54431]: DEBUG nova.network.neutron [None req-d0d58171-f33c-4213-a5d5-cb3cfb0fecd9 tempest-ServerDiagnosticsNegativeTest-1650443261 tempest-ServerDiagnosticsNegativeTest-1650443261-project-member] [instance: 7661165d-2731-456e-af25-da13c68fd9fb] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 628.614045] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 628.625692] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 628.625988] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 628.626236] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 628.664811] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 628.664899] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 628.665166] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 628.665745] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 628.665822] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 628.666596] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 628.676250] nova-conductor[54431]: DEBUG nova.quota [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Getting quotas for project 74849557e5494e259bf7824762dc75a3. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 628.679982] nova-conductor[54431]: DEBUG nova.quota [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Getting quotas for user 4b2a770097844555942cf8293c401aaf and project 74849557e5494e259bf7824762dc75a3. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 628.685750] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] [instance: 78a83324-36da-457e-a78d-c82d0a722015] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 628.686413] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 628.686714] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 628.686971] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 628.691506] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] [instance: 78a83324-36da-457e-a78d-c82d0a722015] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 628.692177] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 628.692383] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 628.692552] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 628.708749] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 628.709067] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 628.709446] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 629.395131] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 629.407517] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 629.407746] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 629.408406] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 629.451048] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 629.451316] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 629.451535] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 629.451960] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 629.452193] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 629.452366] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 629.463269] nova-conductor[54432]: DEBUG nova.quota [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Getting quotas for project 9c30565c40a845588906f604e8cb0e00. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 629.469770] nova-conductor[54432]: DEBUG nova.quota [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Getting quotas for user 9ad60a9ead734f08aa163c15c9a6b81d and project 9c30565c40a845588906f604e8cb0e00. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 629.474918] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] [instance: dee17e73-22db-48ff-b0b0-ec7bec850c48] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 629.475388] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 629.475686] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 629.475885] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 629.482839] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] [instance: dee17e73-22db-48ff-b0b0-ec7bec850c48] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 629.482839] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 629.483124] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 629.483124] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 629.504714] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 629.505541] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 629.505541] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 631.470457] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] [instance: 037217df-a3ed-4cef-9478-a9cc576a2f7d] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 965c7a6a-7b9b-49b0-8f72-a6eea9ca2675, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 037217df-a3ed-4cef-9478-a9cc576a2f7d was re-scheduled: Binding failed for port 965c7a6a-7b9b-49b0-8f72-a6eea9ca2675, please check neutron logs for more information.\n'] [ 631.471133] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 631.471377] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 037217df-a3ed-4cef-9478-a9cc576a2f7d.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 037217df-a3ed-4cef-9478-a9cc576a2f7d. [ 631.471690] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] [instance: 037217df-a3ed-4cef-9478-a9cc576a2f7d] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 037217df-a3ed-4cef-9478-a9cc576a2f7d. [ 631.503228] nova-conductor[54432]: DEBUG nova.network.neutron [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] [instance: 037217df-a3ed-4cef-9478-a9cc576a2f7d] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 631.532946] nova-conductor[54432]: DEBUG nova.network.neutron [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] [instance: 037217df-a3ed-4cef-9478-a9cc576a2f7d] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 631.537318] nova-conductor[54432]: DEBUG nova.network.neutron [None req-4657df64-e839-4424-bf76-08371706d845 tempest-ServerTagsTestJSON-2002162229 tempest-ServerTagsTestJSON-2002162229-project-member] [instance: 037217df-a3ed-4cef-9478-a9cc576a2f7d] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 635.495460] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=347,nova_cell1:UPDATE=163,nova_cell1:INSERT=21,nova_cell1:SAVEPOINT=36,nova_cell1:RELEASE=36 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 636.022735] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] [instance: 9160ca3a-2aa2-4cfc-9900-3e4c770d2657] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port d22afec3-7f0e-413f-b7ee-8529d26f4c99, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 9160ca3a-2aa2-4cfc-9900-3e4c770d2657 was re-scheduled: Binding failed for port d22afec3-7f0e-413f-b7ee-8529d26f4c99, please check neutron logs for more information.\n'] [ 636.023319] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 636.023547] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9160ca3a-2aa2-4cfc-9900-3e4c770d2657.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9160ca3a-2aa2-4cfc-9900-3e4c770d2657. [ 636.024151] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] [instance: 9160ca3a-2aa2-4cfc-9900-3e4c770d2657] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9160ca3a-2aa2-4cfc-9900-3e4c770d2657. [ 636.048645] nova-conductor[54432]: DEBUG nova.network.neutron [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] [instance: 9160ca3a-2aa2-4cfc-9900-3e4c770d2657] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 636.072378] nova-conductor[54432]: DEBUG nova.network.neutron [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] [instance: 9160ca3a-2aa2-4cfc-9900-3e4c770d2657] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 636.075582] nova-conductor[54432]: DEBUG nova.network.neutron [None req-3cc5e0bb-b5cb-47b7-a1bb-5623b10491b4 tempest-ImagesNegativeTestJSON-1203360277 tempest-ImagesNegativeTestJSON-1203360277-project-member] [instance: 9160ca3a-2aa2-4cfc-9900-3e4c770d2657] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 638.669718] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell0:SELECT=7 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 638.694876] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=21,nova_cell1:SAVEPOINT=7,nova_cell1:INSERT=56,nova_cell1:RELEASE=7 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 638.700968] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=42,nova_api:UPDATE=3,nova_api:DELETE=4 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 638.706093] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=42,nova_api:UPDATE=4,nova_api:DELETE=3 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 639.457071] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell0:SELECT=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 639.485484] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=6,nova_cell1:INSERT=14 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 639.499197] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=15 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 639.672019] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 639.688427] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 639.688870] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 639.689167] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 639.736801] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 639.736801] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 639.736801] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 639.736801] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 639.736801] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 639.736801] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 639.751278] nova-conductor[54432]: DEBUG nova.quota [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Getting quotas for project 076da2170578492082394cfcb4b102b5. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 639.755841] nova-conductor[54432]: DEBUG nova.quota [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Getting quotas for user a15aadeb30a04c09b35aae1661b89b66 and project 076da2170578492082394cfcb4b102b5. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 639.764029] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] [instance: f5b10ac7-e20c-439c-bc6f-a3cf2dd26122] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 639.764388] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 639.764631] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 639.764841] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 639.777090] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] [instance: f5b10ac7-e20c-439c-bc6f-a3cf2dd26122] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 639.778080] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 639.778485] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 639.778705] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 639.828507] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 639.828507] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 639.828507] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 640.575064] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 640.598377] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.598655] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.598994] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 640.644050] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.644050] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.644050] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 640.644050] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.644050] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.644050] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 640.658409] nova-conductor[54432]: DEBUG nova.quota [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Getting quotas for project 076da2170578492082394cfcb4b102b5. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 640.670191] nova-conductor[54432]: DEBUG nova.quota [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Getting quotas for user a15aadeb30a04c09b35aae1661b89b66 and project 076da2170578492082394cfcb4b102b5. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 640.680433] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] [instance: ff822880-41dc-429f-80fb-a1ddc0441ea3] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 640.680970] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.681225] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.681444] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 640.689788] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] [instance: ff822880-41dc-429f-80fb-a1ddc0441ea3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 640.690577] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.690798] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.691895] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 640.730689] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 640.730932] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 640.731155] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 641.377552] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=386,nova_cell1:SAVEPOINT=35,nova_cell1:RELEASE=35,nova_cell1:UPDATE=199,nova_cell1:INSERT=29,nova_cell1:DELETE=4 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 643.088877] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Took 0.51 seconds to select destinations for 3 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 643.107376] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.107573] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.107812] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.158205] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.158655] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.158704] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.192391] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.192708] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.192813] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.218072] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.218424] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.218493] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.219052] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.219390] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.219582] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.230827] nova-conductor[54431]: DEBUG nova.quota [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Getting quotas for project dff7a8db03474072848c122886f622d7. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 643.237677] nova-conductor[54431]: DEBUG nova.quota [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Getting quotas for user 2aae85566cb84e41881222ddaa137bd5 and project dff7a8db03474072848c122886f622d7. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 643.243527] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 0224e4da-3079-477f-96d8-671985e85c39] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 643.245305] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.245305] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.245305] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.250585] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 0224e4da-3079-477f-96d8-671985e85c39] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 643.250585] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.250585] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.250585] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.266788] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.266983] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.267148] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.275052] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 477a1892-0316-42ff-8fc8-269239cb2198] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 643.275610] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.275834] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.276020] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.285746] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 477a1892-0316-42ff-8fc8-269239cb2198] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 643.286471] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.286909] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.286909] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.303942] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.304593] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.304593] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.310882] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 399b809e-8926-4348-bfaf-3499de57c1b3] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 643.311454] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.311781] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.311994] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.315784] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 399b809e-8926-4348-bfaf-3499de57c1b3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 643.316615] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.316685] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.318576] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.333722] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.333879] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.338218] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.724531] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 643.752263] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.752263] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.752263] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.818831] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.820804] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.820804] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.820804] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.820804] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.820804] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.829720] nova-conductor[54432]: DEBUG nova.quota [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Getting quotas for project cb9ca6d8c8ae4352bc3c7b1daa7ba288. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 643.836235] nova-conductor[54432]: DEBUG nova.quota [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Getting quotas for user 88c76f25140f4de28bd719de1d5ff616 and project cb9ca6d8c8ae4352bc3c7b1daa7ba288. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 643.844385] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 6e078697-9fc5-4605-8d4a-a47d9331ba9c] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 643.845192] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.845580] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.846022] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.869152] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 6e078697-9fc5-4605-8d4a-a47d9331ba9c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 643.870052] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.870209] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.870965] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 643.896026] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 643.896262] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 643.896429] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 644.669493] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 644.707639] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 644.707864] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 644.708145] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 644.745732] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 644.746485] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 644.746485] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 644.746485] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 644.746670] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 644.746850] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 644.754828] nova-conductor[54431]: DEBUG nova.quota [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Getting quotas for project 4565b45612fb4f7197f883514bf7c028. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 644.757709] nova-conductor[54431]: DEBUG nova.quota [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Getting quotas for user a49582e5ed5448a19188c19f4439a61e and project 4565b45612fb4f7197f883514bf7c028. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 644.763931] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] [instance: e7023ea7-9a39-4e5b-90f6-b2499e06ee9d] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 644.764559] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 644.764813] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 644.765051] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 644.769683] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] [instance: e7023ea7-9a39-4e5b-90f6-b2499e06ee9d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 644.770483] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 644.770780] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 644.771098] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 644.789474] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 644.789820] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 644.790031] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 645.717792] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] [instance: 81b1d279-dfad-4a2c-bcfb-043b13d77af6] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port c853a44f-5ddf-4d9b-b501-5cf4d5772cd4, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 81b1d279-dfad-4a2c-bcfb-043b13d77af6 was re-scheduled: Binding failed for port c853a44f-5ddf-4d9b-b501-5cf4d5772cd4, please check neutron logs for more information.\n'] [ 645.718533] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 645.718836] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 81b1d279-dfad-4a2c-bcfb-043b13d77af6.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 81b1d279-dfad-4a2c-bcfb-043b13d77af6. [ 645.719193] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] [instance: 81b1d279-dfad-4a2c-bcfb-043b13d77af6] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 81b1d279-dfad-4a2c-bcfb-043b13d77af6. [ 645.745040] nova-conductor[54431]: DEBUG nova.network.neutron [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] [instance: 81b1d279-dfad-4a2c-bcfb-043b13d77af6] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 645.770164] nova-conductor[54431]: DEBUG nova.network.neutron [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] [instance: 81b1d279-dfad-4a2c-bcfb-043b13d77af6] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 645.774465] nova-conductor[54431]: DEBUG nova.network.neutron [None req-dc10af78-33a4-4793-8b62-5d350cc01a55 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] [instance: 81b1d279-dfad-4a2c-bcfb-043b13d77af6] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 647.477855] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 647.494264] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 647.494459] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 647.494940] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 647.524050] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 647.525028] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 647.525028] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 647.525028] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 647.525028] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 647.525199] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 647.532457] nova-conductor[54432]: DEBUG nova.quota [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Getting quotas for project d6e3f664f5d3403289b586dcceb28424. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 647.534769] nova-conductor[54432]: DEBUG nova.quota [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Getting quotas for user f7f6adeea8054eefaf9d72e4264474c2 and project d6e3f664f5d3403289b586dcceb28424. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 647.539280] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] [instance: ea14c163-3afe-4a8d-8370-43e26ba7af37] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 647.539747] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 647.539950] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 647.540135] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 647.545051] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] [instance: ea14c163-3afe-4a8d-8370-43e26ba7af37] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 647.545423] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 647.545626] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 647.545791] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 647.560504] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 647.562334] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 647.562334] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 652.743336] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] [instance: ccce765d-8df9-4837-b166-71e41bea6de7] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 29b75515-596f-4fac-b874-1afada623805, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ccce765d-8df9-4837-b166-71e41bea6de7 was re-scheduled: Binding failed for port 29b75515-596f-4fac-b874-1afada623805, please check neutron logs for more information.\n'] [ 652.743919] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 652.744175] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ccce765d-8df9-4837-b166-71e41bea6de7.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ccce765d-8df9-4837-b166-71e41bea6de7. [ 652.744391] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] [instance: ccce765d-8df9-4837-b166-71e41bea6de7] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ccce765d-8df9-4837-b166-71e41bea6de7. [ 652.770293] nova-conductor[54431]: DEBUG nova.network.neutron [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] [instance: ccce765d-8df9-4837-b166-71e41bea6de7] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 652.787436] nova-conductor[54431]: DEBUG nova.network.neutron [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] [instance: ccce765d-8df9-4837-b166-71e41bea6de7] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 652.792022] nova-conductor[54431]: DEBUG nova.network.neutron [None req-c65c1bd5-45db-45ef-8992-d75f966ccaf6 tempest-DeleteServersAdminTestJSON-910099737 tempest-DeleteServersAdminTestJSON-910099737-project-member] [instance: ccce765d-8df9-4837-b166-71e41bea6de7] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 654.752904] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell0:SELECT=2 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 654.772828] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=10,nova_cell1:SAVEPOINT=2,nova_cell1:INSERT=30,nova_cell1:RELEASE=2 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 654.785403] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=13,nova_api:DELETE=3,nova_api:UPDATE=3 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 657.266823] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 287c3404-9fc1-4369-b9a4-17da918bf78d] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port fc688df4-d52b-4751-a8c0-854113d5e094, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 287c3404-9fc1-4369-b9a4-17da918bf78d was re-scheduled: Binding failed for port fc688df4-d52b-4751-a8c0-854113d5e094, please check neutron logs for more information.\n'] [ 657.267830] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 657.268112] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 287c3404-9fc1-4369-b9a4-17da918bf78d.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 287c3404-9fc1-4369-b9a4-17da918bf78d. [ 657.268340] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 287c3404-9fc1-4369-b9a4-17da918bf78d] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 287c3404-9fc1-4369-b9a4-17da918bf78d. [ 657.295842] nova-conductor[54431]: DEBUG nova.network.neutron [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 287c3404-9fc1-4369-b9a4-17da918bf78d] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 657.326689] nova-conductor[54431]: DEBUG nova.network.neutron [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 287c3404-9fc1-4369-b9a4-17da918bf78d] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 657.329449] nova-conductor[54431]: DEBUG nova.network.neutron [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 287c3404-9fc1-4369-b9a4-17da918bf78d] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 657.534145] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell0:SELECT=4 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 657.550068] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=12,nova_cell1:SAVEPOINT=2,nova_cell1:INSERT=30,nova_cell1:RELEASE=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 657.558893] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=19,nova_api:UPDATE=3,nova_api:DELETE=3 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 657.559417] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=38,nova_api:UPDATE=3,nova_api:DELETE=3 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 657.601537] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=20,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 658.303090] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 658.316675] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 658.316996] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 658.317156] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 658.353373] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 658.353451] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 658.353616] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 658.353994] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 658.354196] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 658.354357] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 658.366021] nova-conductor[54432]: DEBUG nova.quota [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Getting quotas for project 26789909db1d4930acc0b87358ba159d. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 658.366601] nova-conductor[54432]: DEBUG nova.quota [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Getting quotas for user 30a7d1e49988462ab49214182fc00e5e and project 26789909db1d4930acc0b87358ba159d. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 658.374065] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] [instance: 6fef0de4-89d3-4acb-bfa0-9f67eca9c3f5] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 658.374545] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 658.374749] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 658.374916] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 658.382022] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] [instance: 6fef0de4-89d3-4acb-bfa0-9f67eca9c3f5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 658.382022] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 658.382022] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 658.382022] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 658.401639] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 658.401856] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 658.402037] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 659.628588] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Took 0.51 seconds to select destinations for 2 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 659.641598] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 659.641834] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 659.642016] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 659.672229] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 659.672465] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 659.672634] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 659.699503] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 659.699726] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 659.700196] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 659.700650] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 659.701207] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 659.701207] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 659.709090] nova-conductor[54432]: DEBUG nova.quota [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Getting quotas for project 7fa50ab2c6ba4659afee1f146822a981. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 659.711245] nova-conductor[54432]: DEBUG nova.quota [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Getting quotas for user d896dc782fdf4c13b943c0336d255452 and project 7fa50ab2c6ba4659afee1f146822a981. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 659.716491] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 7bdd4371-b5d5-4053-93b1-75d1c5b9835b] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 659.717030] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 659.717239] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 659.717403] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 659.721882] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 7bdd4371-b5d5-4053-93b1-75d1c5b9835b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 659.722537] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 659.722732] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 659.722896] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 659.737184] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 659.737408] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 659.737576] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 659.744119] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 63b76c39-11ca-4f1e-b336-2caceece1f72] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 659.744539] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 659.744737] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 659.744903] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 659.747461] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 63b76c39-11ca-4f1e-b336-2caceece1f72] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 659.748175] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 659.748442] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 659.748698] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 659.791024] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 659.791262] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 659.791430] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 660.696629] nova-conductor[54431]: DEBUG nova.db.main.api [None req-a6d5335d-2271-419d-aabf-1fbb9fa0c75c tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Created instance_extra for 6680c707-aa59-4e00-a2b1-38d465cc6497 {{(pid=54431) instance_extra_update_by_uuid /opt/stack/nova/nova/db/main/api.py:2551}} [ 663.688451] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] [instance: 2a963e2c-7ee3-41a4-842b-06c151d27d84] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port c45bc645-978c-44dd-b928-4568f73165f7, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 2a963e2c-7ee3-41a4-842b-06c151d27d84 was re-scheduled: Binding failed for port c45bc645-978c-44dd-b928-4568f73165f7, please check neutron logs for more information.\n'] [ 663.715678] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 663.716372] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2a963e2c-7ee3-41a4-842b-06c151d27d84.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2a963e2c-7ee3-41a4-842b-06c151d27d84. [ 663.716780] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] [instance: 2a963e2c-7ee3-41a4-842b-06c151d27d84] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2a963e2c-7ee3-41a4-842b-06c151d27d84. [ 663.743072] nova-conductor[54432]: DEBUG nova.network.neutron [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] [instance: 2a963e2c-7ee3-41a4-842b-06c151d27d84] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 663.766919] nova-conductor[54432]: DEBUG nova.network.neutron [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] [instance: 2a963e2c-7ee3-41a4-842b-06c151d27d84] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 663.769942] nova-conductor[54432]: DEBUG nova.network.neutron [None req-dcd70b61-11cd-42ae-9900-5b578a1e7541 tempest-InstanceActionsTestJSON-1270824579 tempest-InstanceActionsTestJSON-1270824579-project-member] [instance: 2a963e2c-7ee3-41a4-842b-06c151d27d84] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 665.619816] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] [instance: 5e18d3cc-62bf-4f12-8e89-2e1bcad15d49] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port ef639489-5e4d-4391-8f65-bd9436c56073, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 5e18d3cc-62bf-4f12-8e89-2e1bcad15d49 was re-scheduled: Binding failed for port ef639489-5e4d-4391-8f65-bd9436c56073, please check neutron logs for more information.\n'] [ 665.620536] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 665.621256] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5e18d3cc-62bf-4f12-8e89-2e1bcad15d49.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5e18d3cc-62bf-4f12-8e89-2e1bcad15d49. [ 665.621663] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] [instance: 5e18d3cc-62bf-4f12-8e89-2e1bcad15d49] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5e18d3cc-62bf-4f12-8e89-2e1bcad15d49. [ 665.652117] nova-conductor[54432]: DEBUG nova.network.neutron [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] [instance: 5e18d3cc-62bf-4f12-8e89-2e1bcad15d49] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 665.784994] nova-conductor[54432]: DEBUG nova.network.neutron [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] [instance: 5e18d3cc-62bf-4f12-8e89-2e1bcad15d49] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 665.789375] nova-conductor[54432]: DEBUG nova.network.neutron [None req-6647cacc-c07f-470e-9ff7-8dcdbcf14705 tempest-ServerActionsTestOtherB-1271781834 tempest-ServerActionsTestOtherB-1271781834-project-member] [instance: 5e18d3cc-62bf-4f12-8e89-2e1bcad15d49] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 666.641500] nova-conductor[54431]: DEBUG nova.db.main.api [None req-d57fe4e9-5fe7-4a93-a464-e4d5ae092655 tempest-ServersListShow296Test-1004933319 tempest-ServersListShow296Test-1004933319-project-member] Created instance_extra for da2b873c-0cac-4a7d-8639-b13e0f31b7b6 {{(pid=54431) instance_extra_update_by_uuid /opt/stack/nova/nova/db/main/api.py:2551}} [ 668.746787] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 668.761645] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 668.761645] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 668.761645] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 668.807619] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 668.807991] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 668.808209] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 668.808595] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 668.808819] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 668.808995] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 668.821699] nova-conductor[54431]: DEBUG nova.quota [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Getting quotas for project ef347a1e31844b40b18851d15b81dfe4. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 668.826249] nova-conductor[54431]: DEBUG nova.quota [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Getting quotas for user 989895db84ee4df29f4f75493ddfeecd and project ef347a1e31844b40b18851d15b81dfe4. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 668.831722] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] [instance: 008b230e-72b8-43ae-826a-b38111c56e76] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 668.832140] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 668.832353] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 668.832540] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 668.837404] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] [instance: 008b230e-72b8-43ae-826a-b38111c56e76] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 668.838084] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 668.838377] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 668.838475] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 668.854109] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 668.854332] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 668.854488] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-fad62c2d-ad48-4f64-a4e9-e8805476c0e6 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 669.049158] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 669.062307] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 669.062307] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 669.062441] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 669.099756] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 669.099987] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 669.100176] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 669.100548] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 669.100734] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 669.100893] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 669.108340] nova-conductor[54431]: DEBUG nova.quota [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Getting quotas for project ef347a1e31844b40b18851d15b81dfe4. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 669.110763] nova-conductor[54431]: DEBUG nova.quota [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Getting quotas for user 989895db84ee4df29f4f75493ddfeecd and project ef347a1e31844b40b18851d15b81dfe4. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 669.115890] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] [instance: 39bc8b3a-e9d9-48e3-a193-5fbdc2454346] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 669.116359] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 669.116558] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 669.116721] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 669.119267] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] [instance: 39bc8b3a-e9d9-48e3-a193-5fbdc2454346] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 669.119873] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 669.120088] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 669.120258] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 669.134063] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 669.134278] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 669.134444] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4d1b4e6e-33f6-43d4-b753-8d51290553bf tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 669.707650] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell0:SELECT=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 669.751638] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=8,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=22,nova_cell1:RELEASE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 669.786478] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=3,nova_api:DELETE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 669.788036] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 669.791266] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=15,nova_api:DELETE=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 669.812158] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 669.815068] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 669.815068] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 669.849952] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 669.850247] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 669.850457] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 669.850912] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 669.851184] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 669.851396] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 669.863958] nova-conductor[54431]: DEBUG nova.quota [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Getting quotas for project 41b5d55537834f248e192c24f5b727f5. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 669.870991] nova-conductor[54431]: DEBUG nova.quota [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Getting quotas for user a3c1b0760fc54bfc9d599346a204fffe and project 41b5d55537834f248e192c24f5b727f5. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 669.873410] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] [instance: 15f8e04f-8c14-4315-8686-da4db517f7b6] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 669.874538] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 669.874761] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 669.875069] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 669.879377] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] [instance: 15f8e04f-8c14-4315-8686-da4db517f7b6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 669.880038] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 669.880306] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 669.880469] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 669.906009] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 669.906248] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 669.906419] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-290bd42d-eb9d-4e1c-a1ae-5677ee905ec6 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 670.249420] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] [instance: 0464da14-cb61-4f8f-9bd7-16f970aec0e7] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port adea3ab7-3634-4b13-896b-43473ad8f651, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 0464da14-cb61-4f8f-9bd7-16f970aec0e7 was re-scheduled: Binding failed for port adea3ab7-3634-4b13-896b-43473ad8f651, please check neutron logs for more information.\n'] [ 670.251717] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 670.252061] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0464da14-cb61-4f8f-9bd7-16f970aec0e7.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0464da14-cb61-4f8f-9bd7-16f970aec0e7. [ 670.252432] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] [instance: 0464da14-cb61-4f8f-9bd7-16f970aec0e7] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0464da14-cb61-4f8f-9bd7-16f970aec0e7. [ 670.285464] nova-conductor[54431]: DEBUG nova.network.neutron [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] [instance: 0464da14-cb61-4f8f-9bd7-16f970aec0e7] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 670.303348] nova-conductor[54431]: DEBUG nova.network.neutron [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] [instance: 0464da14-cb61-4f8f-9bd7-16f970aec0e7] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 670.306931] nova-conductor[54431]: DEBUG nova.network.neutron [None req-76450f5c-9e2c-4907-924a-246a3ca4ada0 tempest-FloatingIPsAssociationTestJSON-71830957 tempest-FloatingIPsAssociationTestJSON-71830957-project-member] [instance: 0464da14-cb61-4f8f-9bd7-16f970aec0e7] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 674.275451] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 674.290936] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 674.291179] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 674.291349] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 674.319902] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 674.320141] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 674.320312] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 674.320670] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 674.320854] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 674.321028] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 674.332391] nova-conductor[54432]: DEBUG nova.quota [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Getting quotas for project 5bea0056ac8c4a36b8979705ef763b02. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 674.335833] nova-conductor[54432]: DEBUG nova.quota [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Getting quotas for user a4f47b07ce044d2f9165fae7a796bbd4 and project 5bea0056ac8c4a36b8979705ef763b02. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 674.342139] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] [instance: 57d21761-431b-4a41-86eb-038f8c35d8ef] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 674.342874] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 674.342874] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 674.342990] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 674.347415] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] [instance: 57d21761-431b-4a41-86eb-038f8c35d8ef] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 674.348061] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 674.348270] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 674.348436] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 674.365218] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 674.365376] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 674.365558] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3362da7f-2fe9-426a-98e0-39d630918f82 tempest-ServersAaction247Test-801381229 tempest-ServersAaction247Test-801381229-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 679.857753] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell0:SELECT=3 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 679.883030] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=9,nova_cell1:SAVEPOINT=2,nova_cell1:INSERT=23,nova_cell1:RELEASE=2 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 679.896290] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=16,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 679.903509] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=20,nova_api:DELETE=2,nova_api:UPDATE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 684.326226] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell0:SELECT=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 684.350731] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=3,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=8,nova_cell1:RELEASE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 684.355938] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=6 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 684.364026] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=6,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 687.466427] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] [instance: c1cd40be-a66c-4ed9-b329-6d94509b19cf] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port ebcbf4d4-0f22-4534-a39b-616a3c1f921b, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance c1cd40be-a66c-4ed9-b329-6d94509b19cf was re-scheduled: Binding failed for port ebcbf4d4-0f22-4534-a39b-616a3c1f921b, please check neutron logs for more information.\n'] [ 687.467148] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 687.467396] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c1cd40be-a66c-4ed9-b329-6d94509b19cf.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c1cd40be-a66c-4ed9-b329-6d94509b19cf. [ 687.468800] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] [instance: c1cd40be-a66c-4ed9-b329-6d94509b19cf] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c1cd40be-a66c-4ed9-b329-6d94509b19cf. [ 687.502494] nova-conductor[54432]: DEBUG nova.network.neutron [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] [instance: c1cd40be-a66c-4ed9-b329-6d94509b19cf] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 687.526346] nova-conductor[54432]: DEBUG nova.network.neutron [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] [instance: c1cd40be-a66c-4ed9-b329-6d94509b19cf] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 687.530052] nova-conductor[54432]: DEBUG nova.network.neutron [None req-b104f7df-116a-4149-82cb-de7da5ec818d tempest-ServerGroupTestJSON-422224219 tempest-ServerGroupTestJSON-422224219-project-member] [instance: c1cd40be-a66c-4ed9-b329-6d94509b19cf] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 691.838631] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] [instance: e654ab80-4443-4f60-9d51-6730e1907ffe] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 9951d694-f7eb-4c34-8163-1bdd62fb42a3, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance e654ab80-4443-4f60-9d51-6730e1907ffe was re-scheduled: Binding failed for port 9951d694-f7eb-4c34-8163-1bdd62fb42a3, please check neutron logs for more information.\n'] [ 691.839821] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 691.840083] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e654ab80-4443-4f60-9d51-6730e1907ffe.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e654ab80-4443-4f60-9d51-6730e1907ffe. [ 691.840313] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] [instance: e654ab80-4443-4f60-9d51-6730e1907ffe] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e654ab80-4443-4f60-9d51-6730e1907ffe. [ 691.863053] nova-conductor[54431]: DEBUG nova.network.neutron [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] [instance: e654ab80-4443-4f60-9d51-6730e1907ffe] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 691.883294] nova-conductor[54431]: DEBUG nova.network.neutron [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] [instance: e654ab80-4443-4f60-9d51-6730e1907ffe] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 691.887961] nova-conductor[54431]: DEBUG nova.network.neutron [None req-eac4af78-090c-4a1e-bb7b-16a53d00c5ff tempest-MigrationsAdminTest-439471344 tempest-MigrationsAdminTest-439471344-project-member] [instance: e654ab80-4443-4f60-9d51-6730e1907ffe] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 695.598371] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=294,nova_cell1:UPDATE=131,nova_cell1:SAVEPOINT=36,nova_cell1:RELEASE=34,nova_cell1:INSERT=20,nova_cell1:ROLLBACK=2 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 696.305928] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] [instance: 28832fde-2434-4d5a-ae62-550b8c0eb829] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 94469936-9e70-45b5-bbde-a295c9e733a7, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 28832fde-2434-4d5a-ae62-550b8c0eb829 was re-scheduled: Binding failed for port 94469936-9e70-45b5-bbde-a295c9e733a7, please check neutron logs for more information.\n'] [ 696.306494] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 696.306750] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 28832fde-2434-4d5a-ae62-550b8c0eb829.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 28832fde-2434-4d5a-ae62-550b8c0eb829. [ 696.306969] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] [instance: 28832fde-2434-4d5a-ae62-550b8c0eb829] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 28832fde-2434-4d5a-ae62-550b8c0eb829. [ 696.328846] nova-conductor[54432]: DEBUG nova.network.neutron [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] [instance: 28832fde-2434-4d5a-ae62-550b8c0eb829] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 696.344614] nova-conductor[54432]: DEBUG nova.network.neutron [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] [instance: 28832fde-2434-4d5a-ae62-550b8c0eb829] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 696.348139] nova-conductor[54432]: DEBUG nova.network.neutron [None req-90b2d7e4-f5f9-4c6a-b2e4-0fe27ec60361 tempest-ServerRescueTestJSON-1614811012 tempest-ServerRescueTestJSON-1614811012-project-member] [instance: 28832fde-2434-4d5a-ae62-550b8c0eb829] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 700.866019] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 700.878333] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 700.878643] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 700.878897] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 700.931930] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 700.932191] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 700.932363] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 700.932733] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 700.932910] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 700.933083] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 700.950339] nova-conductor[54431]: DEBUG nova.quota [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Getting quotas for project 4f1e8a17916f4b179a86748cb34ec51d. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 700.953709] nova-conductor[54431]: DEBUG nova.quota [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Getting quotas for user 2fedbea4e5b945918a1e906af369211f and project 4f1e8a17916f4b179a86748cb34ec51d. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 700.959165] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] [instance: 261cbefa-fb2d-48da-a4fe-80b744a931f6] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 700.959606] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 700.959817] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 700.959980] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 700.967243] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] [instance: 261cbefa-fb2d-48da-a4fe-80b744a931f6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 700.967243] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 700.967243] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 700.967243] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 700.986151] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 700.986151] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 700.986151] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 701.108703] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 701.120827] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 701.123029] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 701.123029] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 701.145934] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 701.146383] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 701.146702] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 701.149020] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 701.149020] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 701.149020] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 701.155826] nova-conductor[54431]: DEBUG nova.quota [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Getting quotas for project 4f1e8a17916f4b179a86748cb34ec51d. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 701.158359] nova-conductor[54431]: DEBUG nova.quota [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Getting quotas for user 2fedbea4e5b945918a1e906af369211f and project 4f1e8a17916f4b179a86748cb34ec51d. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 701.165616] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] [instance: 49f9108c-7256-4a2c-9ffd-a6d041a180e4] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 701.165616] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 701.165616] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 701.165616] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 701.170996] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] [instance: 49f9108c-7256-4a2c-9ffd-a6d041a180e4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 701.170996] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 701.170996] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 701.170996] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 701.185227] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 701.185610] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 701.185933] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 701.478282] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=295,nova_cell1:SAVEPOINT=24,nova_cell1:RELEASE=24,nova_cell1:UPDATE=141,nova_cell1:INSERT=22 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 702.658404] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 702.672632] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 702.672728] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 702.672852] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 702.718615] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 702.718886] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 702.719059] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 702.719474] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 702.719662] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 702.719822] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 702.728554] nova-conductor[54431]: DEBUG nova.quota [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Getting quotas for project 5bbbd84912164655851b6e29356849c5. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 702.732671] nova-conductor[54431]: DEBUG nova.quota [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Getting quotas for user ea369595aaf8424e88ef45f27900f1f2 and project 5bbbd84912164655851b6e29356849c5. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 702.740074] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] [instance: d650a1a5-3706-4682-a813-f85ea23098e2] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 702.740585] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 702.740796] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 702.740961] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 702.745725] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] [instance: d650a1a5-3706-4682-a813-f85ea23098e2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 702.746450] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 702.747128] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 702.747128] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 702.772150] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 702.772150] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 702.772150] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 705.494822] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] [instance: 873731fb-8d97-41e7-830b-b9ae4281a737] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 1dd261e2-15db-4d61-8991-84f77bee309f, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 873731fb-8d97-41e7-830b-b9ae4281a737 was re-scheduled: Binding failed for port 1dd261e2-15db-4d61-8991-84f77bee309f, please check neutron logs for more information.\n'] [ 705.495866] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 705.496120] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 873731fb-8d97-41e7-830b-b9ae4281a737.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 873731fb-8d97-41e7-830b-b9ae4281a737. [ 705.496441] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] [instance: 873731fb-8d97-41e7-830b-b9ae4281a737] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 873731fb-8d97-41e7-830b-b9ae4281a737. [ 705.519960] nova-conductor[54431]: DEBUG nova.network.neutron [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] [instance: 873731fb-8d97-41e7-830b-b9ae4281a737] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 705.538353] nova-conductor[54431]: DEBUG nova.network.neutron [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] [instance: 873731fb-8d97-41e7-830b-b9ae4281a737] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 705.541615] nova-conductor[54431]: DEBUG nova.network.neutron [None req-25c0b96c-3e18-4388-8cdb-650d7a088322 tempest-ServerActionsTestOtherA-1189380881 tempest-ServerActionsTestOtherA-1189380881-project-member] [instance: 873731fb-8d97-41e7-830b-b9ae4281a737] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 712.638277] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 712.657198] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 712.657478] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 712.657659] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 712.701072] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 712.701308] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 712.701561] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 712.701845] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 712.702492] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 712.702492] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 712.711916] nova-conductor[54431]: DEBUG nova.quota [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Getting quotas for project d1229f15ad8648da9cf8a3537795d9fa. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 712.714118] nova-conductor[54431]: DEBUG nova.quota [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Getting quotas for user 65a9f926ab8643c88b9251aa1ceb949e and project d1229f15ad8648da9cf8a3537795d9fa. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 712.719433] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] [instance: 60a4b8c3-9dc4-4867-adca-4503e61ce237] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 712.719943] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 712.720175] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 712.720392] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 712.725436] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] [instance: 60a4b8c3-9dc4-4867-adca-4503e61ce237] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 712.726645] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 712.726645] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 712.726645] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 712.744935] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 712.745047] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 712.745226] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 713.067609] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 713.083695] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 713.084667] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 713.084667] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 713.126335] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] [instance: 9ca13604-2b1b-4d47-8a23-8137384a87a6] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 6a63349e-54c1-4d95-a74c-8515707dd186, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 9ca13604-2b1b-4d47-8a23-8137384a87a6 was re-scheduled: Binding failed for port 6a63349e-54c1-4d95-a74c-8515707dd186, please check neutron logs for more information.\n'] [ 713.127361] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 713.127361] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9ca13604-2b1b-4d47-8a23-8137384a87a6.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9ca13604-2b1b-4d47-8a23-8137384a87a6. [ 713.128042] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] [instance: 9ca13604-2b1b-4d47-8a23-8137384a87a6] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9ca13604-2b1b-4d47-8a23-8137384a87a6. [ 713.132047] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 713.132047] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 713.132047] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 713.132047] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 713.132047] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 713.132211] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 713.150428] nova-conductor[54432]: DEBUG nova.quota [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Getting quotas for project d1229f15ad8648da9cf8a3537795d9fa. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 713.153929] nova-conductor[54432]: DEBUG nova.quota [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Getting quotas for user 65a9f926ab8643c88b9251aa1ceb949e and project d1229f15ad8648da9cf8a3537795d9fa. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 713.162983] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] [instance: ad749135-08c1-48ca-af56-300d6e796012] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 713.163487] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 713.163702] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 713.164032] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 713.164192] nova-conductor[54431]: DEBUG nova.network.neutron [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] [instance: 9ca13604-2b1b-4d47-8a23-8137384a87a6] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 713.170622] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] [instance: ad749135-08c1-48ca-af56-300d6e796012] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 713.173352] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 713.173352] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 713.173352] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 713.182069] nova-conductor[54431]: DEBUG nova.network.neutron [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] [instance: 9ca13604-2b1b-4d47-8a23-8137384a87a6] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 713.185890] nova-conductor[54431]: DEBUG nova.network.neutron [None req-089bfebb-80bb-4d04-8f3a-fa0867aafd7e tempest-SecurityGroupsTestJSON-668110514 tempest-SecurityGroupsTestJSON-668110514-project-member] [instance: 9ca13604-2b1b-4d47-8a23-8137384a87a6] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 713.194979] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 713.195233] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 713.195404] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 717.797973] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] [instance: 3b5fb42c-8a81-4575-ba8f-424a101d11ae] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 39337e32-ee39-47e5-ad79-389ade158ef2, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 3b5fb42c-8a81-4575-ba8f-424a101d11ae was re-scheduled: Binding failed for port 39337e32-ee39-47e5-ad79-389ade158ef2, please check neutron logs for more information.\n'] [ 717.798565] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 717.798837] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3b5fb42c-8a81-4575-ba8f-424a101d11ae.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3b5fb42c-8a81-4575-ba8f-424a101d11ae. [ 717.799125] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] [instance: 3b5fb42c-8a81-4575-ba8f-424a101d11ae] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3b5fb42c-8a81-4575-ba8f-424a101d11ae. [ 717.839359] nova-conductor[54431]: DEBUG nova.network.neutron [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] [instance: 3b5fb42c-8a81-4575-ba8f-424a101d11ae] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 717.863534] nova-conductor[54431]: DEBUG nova.network.neutron [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] [instance: 3b5fb42c-8a81-4575-ba8f-424a101d11ae] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 717.867228] nova-conductor[54431]: DEBUG nova.network.neutron [None req-e789abf5-25c9-45bc-8416-5658e6c30b38 tempest-ServersTestBootFromVolume-1640696889 tempest-ServersTestBootFromVolume-1640696889-project-member] [instance: 3b5fb42c-8a81-4575-ba8f-424a101d11ae] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 722.138880] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 722.152602] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 722.153229] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 722.153229] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 722.191721] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 722.192159] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 722.192246] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 722.192637] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 722.192719] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 722.192866] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 722.202487] nova-conductor[54431]: DEBUG nova.quota [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Getting quotas for project d5be96f3d8334da791f2caae93e0624e. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 722.206880] nova-conductor[54431]: DEBUG nova.quota [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Getting quotas for user 4de905221d184f00a5579c750ca56325 and project d5be96f3d8334da791f2caae93e0624e. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 722.215476] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] [instance: 2a1c52ab-8ee6-4b63-ad04-2450a60262d0] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 722.215936] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 722.216304] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 722.216304] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 722.221379] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] [instance: 2a1c52ab-8ee6-4b63-ad04-2450a60262d0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 722.222090] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 722.222309] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 722.222479] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 722.238116] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 722.238391] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 722.238523] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 722.481032] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] [instance: 0899249c-880c-48de-b2c3-2c407115463e] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port ddd7b2bb-0a7c-4f1f-ad88-162c46df78b1, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 0899249c-880c-48de-b2c3-2c407115463e was re-scheduled: Binding failed for port ddd7b2bb-0a7c-4f1f-ad88-162c46df78b1, please check neutron logs for more information.\n'] [ 722.481032] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 722.481032] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0899249c-880c-48de-b2c3-2c407115463e.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0899249c-880c-48de-b2c3-2c407115463e. [ 722.481490] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] [instance: 0899249c-880c-48de-b2c3-2c407115463e] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0899249c-880c-48de-b2c3-2c407115463e. [ 722.504312] nova-conductor[54432]: DEBUG nova.network.neutron [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] [instance: 0899249c-880c-48de-b2c3-2c407115463e] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 722.518914] nova-conductor[54432]: DEBUG nova.network.neutron [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] [instance: 0899249c-880c-48de-b2c3-2c407115463e] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 722.522439] nova-conductor[54432]: DEBUG nova.network.neutron [None req-791139e3-6020-4fd8-9217-e882bd37d858 tempest-InstanceActionsNegativeTestJSON-1284918533 tempest-InstanceActionsNegativeTestJSON-1284918533-project-member] [instance: 0899249c-880c-48de-b2c3-2c407115463e] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 723.140750] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell0:SELECT=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 723.175409] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=3,nova_cell1:INSERT=7 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 723.185554] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=4,nova_api:UPDATE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 723.190268] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=8,nova_api:DELETE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 724.616985] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] [instance: a4a9c3d1-70f5-4521-9325-e3a7f6e7de1b] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port da8078a3-f87c-46cd-ad54-61d196be63f0, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance a4a9c3d1-70f5-4521-9325-e3a7f6e7de1b was re-scheduled: Binding failed for port da8078a3-f87c-46cd-ad54-61d196be63f0, please check neutron logs for more information.\n'] [ 724.617505] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 724.617741] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a4a9c3d1-70f5-4521-9325-e3a7f6e7de1b.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a4a9c3d1-70f5-4521-9325-e3a7f6e7de1b. [ 724.618097] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] [instance: a4a9c3d1-70f5-4521-9325-e3a7f6e7de1b] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a4a9c3d1-70f5-4521-9325-e3a7f6e7de1b. [ 724.645439] nova-conductor[54431]: DEBUG nova.network.neutron [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] [instance: a4a9c3d1-70f5-4521-9325-e3a7f6e7de1b] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 724.668588] nova-conductor[54431]: DEBUG nova.network.neutron [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] [instance: a4a9c3d1-70f5-4521-9325-e3a7f6e7de1b] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 724.672011] nova-conductor[54431]: DEBUG nova.network.neutron [None req-b0c3ddd2-2aa9-4fe5-9b0d-baaff0458097 tempest-ImagesOneServerNegativeTestJSON-1329507913 tempest-ImagesOneServerNegativeTestJSON-1329507913-project-member] [instance: a4a9c3d1-70f5-4521-9325-e3a7f6e7de1b] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 726.731367] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 3d9ac347-ffcc-4840-8090-811dbd2865ef] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 993e00ef-a852-4091-bc3a-7acc78478536, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 3d9ac347-ffcc-4840-8090-811dbd2865ef was re-scheduled: Binding failed for port 993e00ef-a852-4091-bc3a-7acc78478536, please check neutron logs for more information.\n'] [ 726.731367] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 726.731367] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3d9ac347-ffcc-4840-8090-811dbd2865ef.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3d9ac347-ffcc-4840-8090-811dbd2865ef. [ 726.731367] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 3d9ac347-ffcc-4840-8090-811dbd2865ef] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3d9ac347-ffcc-4840-8090-811dbd2865ef. [ 726.763844] nova-conductor[54432]: DEBUG nova.network.neutron [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 3d9ac347-ffcc-4840-8090-811dbd2865ef] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 726.785894] nova-conductor[54432]: DEBUG nova.network.neutron [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 3d9ac347-ffcc-4840-8090-811dbd2865ef] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 726.791451] nova-conductor[54432]: DEBUG nova.network.neutron [None req-051d8d39-24f6-407f-a99a-b7bf55e2f2f8 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 3d9ac347-ffcc-4840-8090-811dbd2865ef] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 729.467334] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 729.480455] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 729.480793] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 729.480978] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 729.517629] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 729.517841] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 729.518025] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 729.518390] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 729.518577] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 729.518739] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 729.525893] nova-conductor[54431]: DEBUG nova.quota [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Getting quotas for project 9512415a4f124e8c9120f6c115f78949. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 729.528306] nova-conductor[54431]: DEBUG nova.quota [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Getting quotas for user a4763a330d70462182e5bdccc2a08c25 and project 9512415a4f124e8c9120f6c115f78949. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 729.533522] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 6da95995-0527-4b05-94dc-a4fcf0ae2f92] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 729.533938] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 729.534156] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 729.534325] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 729.538713] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 6da95995-0527-4b05-94dc-a4fcf0ae2f92] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 729.539365] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 729.539597] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 729.539770] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 729.557697] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 729.557920] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 729.558100] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 731.210026] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] [instance: 3649431e-f856-4a28-9af7-8f101c3fc2d6] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 2d6c29d5-c728-42bd-86a1-1f0309315094, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 3649431e-f856-4a28-9af7-8f101c3fc2d6 was re-scheduled: Binding failed for port 2d6c29d5-c728-42bd-86a1-1f0309315094, please check neutron logs for more information.\n'] [ 731.210674] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 731.210913] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3649431e-f856-4a28-9af7-8f101c3fc2d6.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3649431e-f856-4a28-9af7-8f101c3fc2d6. [ 731.211247] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] [instance: 3649431e-f856-4a28-9af7-8f101c3fc2d6] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3649431e-f856-4a28-9af7-8f101c3fc2d6. [ 731.237179] nova-conductor[54431]: DEBUG nova.network.neutron [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] [instance: 3649431e-f856-4a28-9af7-8f101c3fc2d6] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 731.253284] nova-conductor[54431]: DEBUG nova.network.neutron [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] [instance: 3649431e-f856-4a28-9af7-8f101c3fc2d6] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 731.256173] nova-conductor[54431]: DEBUG nova.network.neutron [None req-85f236cd-415a-4fa9-8043-0de46fbbe27c tempest-ServersTestManualDisk-920724852 tempest-ServersTestManualDisk-920724852-project-member] [instance: 3649431e-f856-4a28-9af7-8f101c3fc2d6] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 733.213984] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 71fa08bc-a5c7-459c-b65b-5dacd9537281] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b10bfa9a-fa7b-425e-b992-d5db1489cc77, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 71fa08bc-a5c7-459c-b65b-5dacd9537281 was re-scheduled: Binding failed for port b10bfa9a-fa7b-425e-b992-d5db1489cc77, please check neutron logs for more information.\n'] [ 733.213984] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 733.213984] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 71fa08bc-a5c7-459c-b65b-5dacd9537281.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 71fa08bc-a5c7-459c-b65b-5dacd9537281. [ 733.213984] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 71fa08bc-a5c7-459c-b65b-5dacd9537281] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 71fa08bc-a5c7-459c-b65b-5dacd9537281. [ 733.248742] nova-conductor[54431]: DEBUG nova.network.neutron [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 71fa08bc-a5c7-459c-b65b-5dacd9537281] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 733.288268] nova-conductor[54431]: DEBUG nova.network.neutron [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 71fa08bc-a5c7-459c-b65b-5dacd9537281] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 733.292674] nova-conductor[54431]: DEBUG nova.network.neutron [None req-462c970d-8e5c-46b5-a82b-3be7d8a26e3a tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 71fa08bc-a5c7-459c-b65b-5dacd9537281] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 735.338090] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 735.350425] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 735.350624] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 735.350797] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 735.385256] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 735.385546] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 735.385656] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 735.386019] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 735.387531] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 735.387531] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 735.396840] nova-conductor[54431]: DEBUG nova.quota [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Getting quotas for project 3f11318c300c4c308f3f8a14db355954. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 735.399133] nova-conductor[54431]: DEBUG nova.quota [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Getting quotas for user 4762545dc5e14c16b5cbb1bad27d90a2 and project 3f11318c300c4c308f3f8a14db355954. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 735.404784] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 263931cd-b2dc-41bb-8a2b-abf61aadafc9] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 735.405532] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 735.405792] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 735.406018] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 735.409497] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 263931cd-b2dc-41bb-8a2b-abf61aadafc9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 735.410168] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 735.410371] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 735.410536] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 735.425826] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 735.426307] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 735.426307] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 737.689393] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] [instance: 6ae11b59-68af-482a-ad65-543a56d7aaa4] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 89eccc4e-8949-458d-a72c-cb35a3c3f973, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 6ae11b59-68af-482a-ad65-543a56d7aaa4 was re-scheduled: Binding failed for port 89eccc4e-8949-458d-a72c-cb35a3c3f973, please check neutron logs for more information.\n'] [ 737.690968] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 737.691172] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6ae11b59-68af-482a-ad65-543a56d7aaa4.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6ae11b59-68af-482a-ad65-543a56d7aaa4. [ 737.691555] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] [instance: 6ae11b59-68af-482a-ad65-543a56d7aaa4] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6ae11b59-68af-482a-ad65-543a56d7aaa4. [ 737.728796] nova-conductor[54432]: DEBUG nova.network.neutron [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] [instance: 6ae11b59-68af-482a-ad65-543a56d7aaa4] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 737.752800] nova-conductor[54432]: DEBUG nova.network.neutron [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] [instance: 6ae11b59-68af-482a-ad65-543a56d7aaa4] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 737.757109] nova-conductor[54432]: DEBUG nova.network.neutron [None req-b936d681-794c-4669-a040-bf8c73dee3ec tempest-ServerActionsV293TestJSON-154139406 tempest-ServerActionsV293TestJSON-154139406-project-member] [instance: 6ae11b59-68af-482a-ad65-543a56d7aaa4] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 741.700453] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Took 0.52 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 741.728803] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 741.729023] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 741.729191] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 741.770233] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 741.770386] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 741.770754] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 741.771232] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 741.771447] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 741.772104] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 741.785566] nova-conductor[54431]: DEBUG nova.quota [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Getting quotas for project 2fa336a944de4078b3a0ef7578f556d0. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 741.789920] nova-conductor[54431]: DEBUG nova.quota [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Getting quotas for user 108621ddd4b2490db4e763b24a9dca73 and project 2fa336a944de4078b3a0ef7578f556d0. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 741.795522] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] [instance: 20e4ff1b-0b84-477d-a0d0-a85a439449ad] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 741.796487] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 741.796487] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 741.796609] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 741.802160] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] [instance: 20e4ff1b-0b84-477d-a0d0-a85a439449ad] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 741.803032] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 741.803220] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 741.803403] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 741.839167] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 741.839167] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 741.839167] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 742.216576] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] [instance: 6d8b7855-567c-44c3-90ee-fc0daec87f46] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port d4db130b-8be3-4540-8b5d-cec6c5fc0da5, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 6d8b7855-567c-44c3-90ee-fc0daec87f46 was re-scheduled: Binding failed for port d4db130b-8be3-4540-8b5d-cec6c5fc0da5, please check neutron logs for more information.\n'] [ 742.217252] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 742.217514] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6d8b7855-567c-44c3-90ee-fc0daec87f46.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6d8b7855-567c-44c3-90ee-fc0daec87f46. [ 742.217874] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] [instance: 6d8b7855-567c-44c3-90ee-fc0daec87f46] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6d8b7855-567c-44c3-90ee-fc0daec87f46. [ 742.251124] nova-conductor[54431]: DEBUG nova.network.neutron [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] [instance: 6d8b7855-567c-44c3-90ee-fc0daec87f46] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 742.264712] nova-conductor[54431]: DEBUG nova.network.neutron [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] [instance: 6d8b7855-567c-44c3-90ee-fc0daec87f46] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 742.268513] nova-conductor[54431]: DEBUG nova.network.neutron [None req-dc9bd693-02cd-4f3d-ba3c-be92e83d1e51 tempest-ServersV294TestFqdnHostnames-987785393 tempest-ServersV294TestFqdnHostnames-987785393-project-member] [instance: 6d8b7855-567c-44c3-90ee-fc0daec87f46] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 746.696422] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 746.715018] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 746.715018] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 746.715018] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 746.753116] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 746.753352] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 746.753517] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 746.753884] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 746.754193] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 746.754334] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 746.765193] nova-conductor[54432]: DEBUG nova.quota [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Getting quotas for project 967135c837004321824783716458da8e. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 746.766440] nova-conductor[54432]: DEBUG nova.quota [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Getting quotas for user ffc458db78a04358af903d6119c5eac9 and project 967135c837004321824783716458da8e. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 746.784665] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] [instance: 4eda80db-5db7-4fd6-9932-108c22f4a616] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 746.785119] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 746.785320] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 746.785483] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 746.802334] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] [instance: 4eda80db-5db7-4fd6-9932-108c22f4a616] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 746.803078] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 746.803295] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 746.804413] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 746.830012] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 746.830284] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 746.830415] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 747.808933] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 747.825574] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 747.825884] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 747.826046] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 747.864295] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 747.864517] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 747.864738] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 747.865052] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 747.865229] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 747.865380] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 747.876757] nova-conductor[54432]: DEBUG nova.quota [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Getting quotas for project 9056b5a2acfa45a48b5b1530dd9b7860. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 747.879609] nova-conductor[54432]: DEBUG nova.quota [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Getting quotas for user d67d200795f843cc81b10074a30aae4a and project 9056b5a2acfa45a48b5b1530dd9b7860. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 747.888113] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: c1961f60-06d8-44d8-999f-d498c3667adc] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 747.888618] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 747.888822] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 747.888992] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 747.894129] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: c1961f60-06d8-44d8-999f-d498c3667adc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 747.894833] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 747.895000] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 747.895182] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 747.915910] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 747.916151] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 747.916321] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 748.156777] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 748.170146] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.170361] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.170563] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 748.191637] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.191875] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.192111] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 748.192484] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.192670] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.192834] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 748.204141] nova-conductor[54431]: DEBUG nova.quota [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Getting quotas for project 9056b5a2acfa45a48b5b1530dd9b7860. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 748.206220] nova-conductor[54431]: DEBUG nova.quota [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Getting quotas for user d67d200795f843cc81b10074a30aae4a and project 9056b5a2acfa45a48b5b1530dd9b7860. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 748.211473] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: ba8fc3e0-dae8-4910-858b-dab1de72089e] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 748.211890] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.212097] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.212268] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 748.218216] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: ba8fc3e0-dae8-4910-858b-dab1de72089e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 748.219261] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.219465] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.219651] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 748.233234] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.233437] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.235994] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 748.698358] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 748.717593] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.717824] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.718481] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 748.758210] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.758487] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.758697] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 748.759547] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.760121] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.760121] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 748.769975] nova-conductor[54432]: DEBUG nova.quota [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Getting quotas for project 9056b5a2acfa45a48b5b1530dd9b7860. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 748.781876] nova-conductor[54432]: DEBUG nova.quota [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Getting quotas for user d67d200795f843cc81b10074a30aae4a and project 9056b5a2acfa45a48b5b1530dd9b7860. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 748.792137] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: 0eaadb76-e2c7-4e06-b865-065ab0f014ba] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 748.792625] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.792830] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.792994] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 748.812414] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: 0eaadb76-e2c7-4e06-b865-065ab0f014ba] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 748.813197] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.813417] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.813585] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 748.869913] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 748.870235] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 748.870469] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 751.943490] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] [instance: 8a769120-906c-4e7c-ac19-b7895039efc1] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 9625f688-ed1b-4676-a575-789088c0a6e6, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 8a769120-906c-4e7c-ac19-b7895039efc1 was re-scheduled: Binding failed for port 9625f688-ed1b-4676-a575-789088c0a6e6, please check neutron logs for more information.\n'] [ 751.944026] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 751.944249] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8a769120-906c-4e7c-ac19-b7895039efc1.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8a769120-906c-4e7c-ac19-b7895039efc1. [ 751.944632] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] [instance: 8a769120-906c-4e7c-ac19-b7895039efc1] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8a769120-906c-4e7c-ac19-b7895039efc1. [ 751.968252] nova-conductor[54431]: DEBUG nova.network.neutron [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] [instance: 8a769120-906c-4e7c-ac19-b7895039efc1] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 751.983954] nova-conductor[54431]: DEBUG nova.network.neutron [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] [instance: 8a769120-906c-4e7c-ac19-b7895039efc1] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 751.993124] nova-conductor[54431]: DEBUG nova.network.neutron [None req-be438baf-e4ff-4f8b-aca3-6b1d91343a65 tempest-ServerPasswordTestJSON-2127054147 tempest-ServerPasswordTestJSON-2127054147-project-member] [instance: 8a769120-906c-4e7c-ac19-b7895039efc1] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 755.765707] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SAVEPOINT=38,nova_cell1:RELEASE=38,nova_cell1:UPDATE=172,nova_cell1:SELECT=354,nova_cell1:INSERT=22 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 758.197625] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell0:SELECT=9 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 758.222198] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=27,nova_cell1:SAVEPOINT=5,nova_cell1:INSERT=68,nova_cell1:RELEASE=5 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 758.229059] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=49,nova_api:DELETE=4,nova_api:UPDATE=3 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 758.232464] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=59,nova_api:UPDATE=6,nova_api:DELETE=5 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 758.764052] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell0:SELECT=3 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 758.816434] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=9,nova_cell1:SAVEPOINT=2,nova_cell1:INSERT=23,nova_cell1:RELEASE=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 758.864019] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=20,nova_api:UPDATE=3,nova_api:DELETE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 758.867561] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=16,nova_api:DELETE=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 758.949286] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 75b04055-8c7e-4659-9451-ddcd6d39fe2e] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b5066e31-5b07-4469-91f7-8ef4b5ef88da, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 75b04055-8c7e-4659-9451-ddcd6d39fe2e was re-scheduled: Binding failed for port b5066e31-5b07-4469-91f7-8ef4b5ef88da, please check neutron logs for more information.\n'] [ 758.949879] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 758.950114] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 75b04055-8c7e-4659-9451-ddcd6d39fe2e.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 75b04055-8c7e-4659-9451-ddcd6d39fe2e. [ 758.950337] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 75b04055-8c7e-4659-9451-ddcd6d39fe2e] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 75b04055-8c7e-4659-9451-ddcd6d39fe2e. [ 758.979392] nova-conductor[54431]: DEBUG nova.network.neutron [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 75b04055-8c7e-4659-9451-ddcd6d39fe2e] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 759.004245] nova-conductor[54431]: DEBUG nova.network.neutron [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 75b04055-8c7e-4659-9451-ddcd6d39fe2e] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 759.009282] nova-conductor[54431]: DEBUG nova.network.neutron [None req-c035c9c6-f75d-4eea-a7ca-6cd5a5fae1b7 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 75b04055-8c7e-4659-9451-ddcd6d39fe2e] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 760.474202] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 760.487939] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 760.488079] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 760.488247] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 760.516419] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 760.516644] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 760.516813] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 760.517189] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 760.517373] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 760.517531] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 760.524757] nova-conductor[54431]: DEBUG nova.quota [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Getting quotas for project e444950a637b4fdab2edb1d62aac4be5. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 760.526826] nova-conductor[54431]: DEBUG nova.quota [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Getting quotas for user c92b9d37f796479ca3b6d9f0ed1789c8 and project e444950a637b4fdab2edb1d62aac4be5. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 760.531854] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 764ef9ab-22ae-45cd-8b5d-76a63346d93c] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 760.532279] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 760.532472] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 760.532631] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 760.537556] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 764ef9ab-22ae-45cd-8b5d-76a63346d93c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 760.538178] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 760.538375] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 760.538538] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 760.552184] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 760.552355] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 760.552523] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 761.591453] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:UPDATE=173,nova_cell1:SELECT=371,nova_cell1:SAVEPOINT=38,nova_cell1:RELEASE=38,nova_cell1:INSERT=25 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 763.359665] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] [instance: 78a83324-36da-457e-a78d-c82d0a722015] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port ab3d96c4-7e66-4d1c-9ee7-2acc85624f4a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 78a83324-36da-457e-a78d-c82d0a722015 was re-scheduled: Binding failed for port ab3d96c4-7e66-4d1c-9ee7-2acc85624f4a, please check neutron logs for more information.\n'] [ 763.360281] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 763.360546] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 78a83324-36da-457e-a78d-c82d0a722015.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 78a83324-36da-457e-a78d-c82d0a722015. [ 763.361154] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] [instance: 78a83324-36da-457e-a78d-c82d0a722015] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 78a83324-36da-457e-a78d-c82d0a722015. [ 763.383101] nova-conductor[54431]: DEBUG nova.network.neutron [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] [instance: 78a83324-36da-457e-a78d-c82d0a722015] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 763.405813] nova-conductor[54431]: DEBUG nova.network.neutron [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] [instance: 78a83324-36da-457e-a78d-c82d0a722015] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 763.409669] nova-conductor[54431]: DEBUG nova.network.neutron [None req-424db3ec-4aa1-4116-b20a-8ebc29792225 tempest-AttachInterfacesV270Test-336752351 tempest-AttachInterfacesV270Test-336752351-project-member] [instance: 78a83324-36da-457e-a78d-c82d0a722015] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 767.770878] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] [instance: dee17e73-22db-48ff-b0b0-ec7bec850c48] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 724ced36-552b-431b-84db-b53ae96f6fda, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance dee17e73-22db-48ff-b0b0-ec7bec850c48 was re-scheduled: Binding failed for port 724ced36-552b-431b-84db-b53ae96f6fda, please check neutron logs for more information.\n'] [ 767.771465] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 767.771733] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance dee17e73-22db-48ff-b0b0-ec7bec850c48.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance dee17e73-22db-48ff-b0b0-ec7bec850c48. [ 767.772014] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] [instance: dee17e73-22db-48ff-b0b0-ec7bec850c48] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance dee17e73-22db-48ff-b0b0-ec7bec850c48. [ 767.791485] nova-conductor[54431]: DEBUG nova.network.neutron [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] [instance: dee17e73-22db-48ff-b0b0-ec7bec850c48] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 767.806792] nova-conductor[54431]: DEBUG nova.network.neutron [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] [instance: dee17e73-22db-48ff-b0b0-ec7bec850c48] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 767.810236] nova-conductor[54431]: DEBUG nova.network.neutron [None req-ec648a66-2364-4d11-829d-c9d9197b7972 tempest-ServersTestMultiNic-232926952 tempest-ServersTestMultiNic-232926952-project-member] [instance: dee17e73-22db-48ff-b0b0-ec7bec850c48] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 769.701455] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] [instance: f5b10ac7-e20c-439c-bc6f-a3cf2dd26122] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port f2efa323-2a49-4acf-84ce-6cd3af0dcd52, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance f5b10ac7-e20c-439c-bc6f-a3cf2dd26122 was re-scheduled: Binding failed for port f2efa323-2a49-4acf-84ce-6cd3af0dcd52, please check neutron logs for more information.\n'] [ 769.702091] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 769.702319] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f5b10ac7-e20c-439c-bc6f-a3cf2dd26122.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f5b10ac7-e20c-439c-bc6f-a3cf2dd26122. [ 769.702766] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] [instance: f5b10ac7-e20c-439c-bc6f-a3cf2dd26122] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f5b10ac7-e20c-439c-bc6f-a3cf2dd26122. [ 769.734819] nova-conductor[54432]: DEBUG nova.network.neutron [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] [instance: f5b10ac7-e20c-439c-bc6f-a3cf2dd26122] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 769.752251] nova-conductor[54432]: DEBUG nova.network.neutron [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] [instance: f5b10ac7-e20c-439c-bc6f-a3cf2dd26122] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 769.755544] nova-conductor[54432]: DEBUG nova.network.neutron [None req-f9faa728-de6b-47a7-88de-8c3bf7cf5cb9 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] [instance: f5b10ac7-e20c-439c-bc6f-a3cf2dd26122] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 770.521296] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell0:SELECT=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 770.540930] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=3,nova_cell1:INSERT=7 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 770.548864] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=7 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 770.551099] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=5,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 771.627088] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] [instance: ff822880-41dc-429f-80fb-a1ddc0441ea3] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 9cdf2850-ba3f-49bf-afc4-6b926de1325b, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ff822880-41dc-429f-80fb-a1ddc0441ea3 was re-scheduled: Binding failed for port 9cdf2850-ba3f-49bf-afc4-6b926de1325b, please check neutron logs for more information.\n'] [ 771.627665] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 771.627890] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ff822880-41dc-429f-80fb-a1ddc0441ea3.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ff822880-41dc-429f-80fb-a1ddc0441ea3. [ 771.628114] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] [instance: ff822880-41dc-429f-80fb-a1ddc0441ea3] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ff822880-41dc-429f-80fb-a1ddc0441ea3. [ 771.653530] nova-conductor[54432]: DEBUG nova.network.neutron [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] [instance: ff822880-41dc-429f-80fb-a1ddc0441ea3] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 771.671748] nova-conductor[54432]: DEBUG nova.network.neutron [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] [instance: ff822880-41dc-429f-80fb-a1ddc0441ea3] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 771.675295] nova-conductor[54432]: DEBUG nova.network.neutron [None req-6a118b60-2d9d-450e-8ffc-51dc75500cd0 tempest-ServersAdminTestJSON-1716551685 tempest-ServersAdminTestJSON-1716551685-project-member] [instance: ff822880-41dc-429f-80fb-a1ddc0441ea3] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 776.015355] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 0224e4da-3079-477f-96d8-671985e85c39] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 0a9d41a4-fdd1-4ccc-82a7-302751c3e083, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 0224e4da-3079-477f-96d8-671985e85c39 was re-scheduled: Binding failed for port 0a9d41a4-fdd1-4ccc-82a7-302751c3e083, please check neutron logs for more information.\n'] [ 776.017693] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 776.017693] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0224e4da-3079-477f-96d8-671985e85c39.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0224e4da-3079-477f-96d8-671985e85c39. [ 776.017693] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 0224e4da-3079-477f-96d8-671985e85c39] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0224e4da-3079-477f-96d8-671985e85c39. [ 776.043856] nova-conductor[54432]: DEBUG nova.network.neutron [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 0224e4da-3079-477f-96d8-671985e85c39] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 776.060183] nova-conductor[54432]: DEBUG nova.network.neutron [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 0224e4da-3079-477f-96d8-671985e85c39] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 776.063508] nova-conductor[54432]: DEBUG nova.network.neutron [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 0224e4da-3079-477f-96d8-671985e85c39] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 778.007889] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 477a1892-0316-42ff-8fc8-269239cb2198] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 87103792-bec8-416e-89d2-1438ea63251a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 477a1892-0316-42ff-8fc8-269239cb2198 was re-scheduled: Binding failed for port 87103792-bec8-416e-89d2-1438ea63251a, please check neutron logs for more information.\n'] [ 778.008786] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 778.008932] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 477a1892-0316-42ff-8fc8-269239cb2198.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 477a1892-0316-42ff-8fc8-269239cb2198. [ 778.009239] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 477a1892-0316-42ff-8fc8-269239cb2198] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 477a1892-0316-42ff-8fc8-269239cb2198. [ 778.044267] nova-conductor[54431]: DEBUG nova.network.neutron [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 477a1892-0316-42ff-8fc8-269239cb2198] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 778.065415] nova-conductor[54431]: DEBUG nova.network.neutron [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 477a1892-0316-42ff-8fc8-269239cb2198] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 778.068122] nova-conductor[54431]: DEBUG nova.network.neutron [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 477a1892-0316-42ff-8fc8-269239cb2198] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 779.518292] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 779.539460] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 779.539460] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 779.539460] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 779.571801] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 779.572490] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 779.572490] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 779.572660] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 779.572815] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 779.572994] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 779.583062] nova-conductor[54431]: DEBUG nova.quota [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Getting quotas for project c9f95d7e954f4413b01aad519c498967. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 779.585763] nova-conductor[54431]: DEBUG nova.quota [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Getting quotas for user 7da3078e8bb5444c96e007cfc2d57952 and project c9f95d7e954f4413b01aad519c498967. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 779.591118] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: a92509e3-d5b1-41d3-b9af-fa0af8a956c9] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 779.591547] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 779.591747] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 779.591918] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 779.603287] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: a92509e3-d5b1-41d3-b9af-fa0af8a956c9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 779.603660] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 779.603882] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 779.604067] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 779.619222] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 779.619374] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 779.619530] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 789.576790] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell0:SELECT=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 789.606952] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=3,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=8,nova_cell1:RELEASE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 789.615029] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=7 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 789.616982] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=5,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 795.197956] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 399b809e-8926-4348-bfaf-3499de57c1b3] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 7aef66d5-4e71-41da-a7f8-5ea10eb75abd, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 399b809e-8926-4348-bfaf-3499de57c1b3 was re-scheduled: Binding failed for port 7aef66d5-4e71-41da-a7f8-5ea10eb75abd, please check neutron logs for more information.\n'] [ 795.198530] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 795.198755] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 399b809e-8926-4348-bfaf-3499de57c1b3.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 399b809e-8926-4348-bfaf-3499de57c1b3. [ 795.198966] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 399b809e-8926-4348-bfaf-3499de57c1b3] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 399b809e-8926-4348-bfaf-3499de57c1b3. [ 795.221832] nova-conductor[54432]: DEBUG nova.network.neutron [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 399b809e-8926-4348-bfaf-3499de57c1b3] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 795.247562] nova-conductor[54432]: DEBUG nova.network.neutron [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 399b809e-8926-4348-bfaf-3499de57c1b3] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 795.250431] nova-conductor[54432]: DEBUG nova.network.neutron [None req-460c06c1-91d0-4b94-a895-15168827dcea tempest-ListServersNegativeTestJSON-1227509665 tempest-ListServersNegativeTestJSON-1227509665-project-member] [instance: 399b809e-8926-4348-bfaf-3499de57c1b3] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 799.557842] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 6e078697-9fc5-4605-8d4a-a47d9331ba9c] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 8d274454-fb43-45a5-9745-eda3d5e00015, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 6e078697-9fc5-4605-8d4a-a47d9331ba9c was re-scheduled: Binding failed for port 8d274454-fb43-45a5-9745-eda3d5e00015, please check neutron logs for more information.\n'] [ 799.558497] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 799.558720] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6e078697-9fc5-4605-8d4a-a47d9331ba9c.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6e078697-9fc5-4605-8d4a-a47d9331ba9c. [ 799.558928] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 6e078697-9fc5-4605-8d4a-a47d9331ba9c] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6e078697-9fc5-4605-8d4a-a47d9331ba9c. [ 799.584375] nova-conductor[54432]: DEBUG nova.network.neutron [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 6e078697-9fc5-4605-8d4a-a47d9331ba9c] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 799.622852] nova-conductor[54432]: DEBUG nova.network.neutron [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 6e078697-9fc5-4605-8d4a-a47d9331ba9c] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 799.626410] nova-conductor[54432]: DEBUG nova.network.neutron [None req-a1f3e394-71e3-47cd-ac1c-b6f0d72e7ecd tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 6e078697-9fc5-4605-8d4a-a47d9331ba9c] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 803.535584] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 803.551104] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 803.551104] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 803.551104] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 803.587418] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 803.587548] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 803.587855] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 803.588277] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 803.588551] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 803.588773] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 803.600949] nova-conductor[54432]: DEBUG nova.quota [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Getting quotas for project cb9ca6d8c8ae4352bc3c7b1daa7ba288. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 803.603235] nova-conductor[54432]: DEBUG nova.quota [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Getting quotas for user 88c76f25140f4de28bd719de1d5ff616 and project cb9ca6d8c8ae4352bc3c7b1daa7ba288. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 803.611284] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 770160d6-4681-40e0-a812-5c491b804da4] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 803.611732] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 803.611939] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 803.612121] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 803.617957] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 770160d6-4681-40e0-a812-5c491b804da4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 803.618667] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 803.618906] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 803.619049] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 803.636547] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 803.638923] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 803.638923] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 803.697260] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 803.707740] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 803.707969] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 803.708148] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 803.743443] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 803.743668] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 803.743836] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 803.744221] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 803.744406] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 803.744560] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 803.752484] nova-conductor[54432]: DEBUG nova.quota [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Getting quotas for project ff1a08f012bf4871bfd8f405acc32220. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 803.754710] nova-conductor[54432]: DEBUG nova.quota [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Getting quotas for user dc0325ff468f4160a86c0a1abb942b6f and project ff1a08f012bf4871bfd8f405acc32220. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 803.760429] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] [instance: cbe51201-c036-4f90-9372-8cc2d033f7a2] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 803.761064] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 803.761295] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 803.761507] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 803.764509] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] [instance: cbe51201-c036-4f90-9372-8cc2d033f7a2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 803.765160] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 803.765361] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 803.765523] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 803.779974] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 803.780215] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 803.780381] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 809.585787] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] [instance: e7023ea7-9a39-4e5b-90f6-b2499e06ee9d] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 7b317662-2319-4c6a-8a87-72b6d90c23aa, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance e7023ea7-9a39-4e5b-90f6-b2499e06ee9d was re-scheduled: Binding failed for port 7b317662-2319-4c6a-8a87-72b6d90c23aa, please check neutron logs for more information.\n'] [ 809.586361] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 809.586607] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e7023ea7-9a39-4e5b-90f6-b2499e06ee9d.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e7023ea7-9a39-4e5b-90f6-b2499e06ee9d. [ 809.586974] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] [instance: e7023ea7-9a39-4e5b-90f6-b2499e06ee9d] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e7023ea7-9a39-4e5b-90f6-b2499e06ee9d. [ 809.609267] nova-conductor[54431]: DEBUG nova.network.neutron [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] [instance: e7023ea7-9a39-4e5b-90f6-b2499e06ee9d] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 809.635822] nova-conductor[54431]: DEBUG nova.network.neutron [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] [instance: e7023ea7-9a39-4e5b-90f6-b2499e06ee9d] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 809.639160] nova-conductor[54431]: DEBUG nova.network.neutron [None req-0b950cc2-a421-4531-a439-b37e9bd695fc tempest-ImagesOneServerTestJSON-628175624 tempest-ImagesOneServerTestJSON-628175624-project-member] [instance: e7023ea7-9a39-4e5b-90f6-b2499e06ee9d] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 813.748928] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell0:SELECT=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 813.767368] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=6,nova_cell1:INSERT=15,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 813.775367] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=15,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 813.777610] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 815.858479] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=291,nova_cell1:UPDATE=140,nova_cell1:INSERT=22,nova_cell1:SAVEPOINT=31,nova_cell1:RELEASE=31 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 816.391254] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] [instance: ea14c163-3afe-4a8d-8370-43e26ba7af37] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 0ae0b0d1-4514-4899-a479-3423a0b56ede, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ea14c163-3afe-4a8d-8370-43e26ba7af37 was re-scheduled: Binding failed for port 0ae0b0d1-4514-4899-a479-3423a0b56ede, please check neutron logs for more information.\n'] [ 816.391472] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 816.391523] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ea14c163-3afe-4a8d-8370-43e26ba7af37.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ea14c163-3afe-4a8d-8370-43e26ba7af37. [ 816.391763] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] [instance: ea14c163-3afe-4a8d-8370-43e26ba7af37] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ea14c163-3afe-4a8d-8370-43e26ba7af37. [ 816.420361] nova-conductor[54431]: DEBUG nova.network.neutron [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] [instance: ea14c163-3afe-4a8d-8370-43e26ba7af37] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 816.436118] nova-conductor[54431]: DEBUG nova.network.neutron [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] [instance: ea14c163-3afe-4a8d-8370-43e26ba7af37] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 816.438648] nova-conductor[54431]: DEBUG nova.network.neutron [None req-254678da-4779-4b4d-b55f-2be552d303d4 tempest-VolumesAdminNegativeTest-61611212 tempest-VolumesAdminNegativeTest-61611212-project-member] [instance: ea14c163-3afe-4a8d-8370-43e26ba7af37] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 817.750690] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-9cebef17-ae3b-41bf-bdeb-f097967f0c03 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Targeting cell 8fb5f300-ab58-4d7c-afe8-8d0b4be88dee(cell1) for conductor method rebuild_instance {{(pid=54431) wrapper /opt/stack/nova/nova/conductor/manager.py:95}} [ 817.750969] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-9cebef17-ae3b-41bf-bdeb-f097967f0c03 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 817.751185] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-9cebef17-ae3b-41bf-bdeb-f097967f0c03 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 817.751322] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-9cebef17-ae3b-41bf-bdeb-f097967f0c03 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 817.763918] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-9cebef17-ae3b-41bf-bdeb-f097967f0c03 tempest-ServerShowV247Test-575892637 tempest-ServerShowV247Test-575892637-project-member] [instance: 39bc8b3a-e9d9-48e3-a193-5fbdc2454346] No migration record for the rebuild/evacuate request. {{(pid=54431) rebuild_instance /opt/stack/nova/nova/conductor/manager.py:1227}} [ 819.421030] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-72fbff57-f32c-4a3d-8870-46f30b364e69 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Targeting cell 8fb5f300-ab58-4d7c-afe8-8d0b4be88dee(cell1) for conductor method rebuild_instance {{(pid=54432) wrapper /opt/stack/nova/nova/conductor/manager.py:95}} [ 819.421271] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-72fbff57-f32c-4a3d-8870-46f30b364e69 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 819.421485] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-72fbff57-f32c-4a3d-8870-46f30b364e69 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 819.421653] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-72fbff57-f32c-4a3d-8870-46f30b364e69 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 819.432575] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-72fbff57-f32c-4a3d-8870-46f30b364e69 tempest-ServerShowV254Test-642463904 tempest-ServerShowV254Test-642463904-project-member] [instance: 15f8e04f-8c14-4315-8686-da4db517f7b6] No migration record for the rebuild/evacuate request. {{(pid=54432) rebuild_instance /opt/stack/nova/nova/conductor/manager.py:1227}} [ 820.650652] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] [instance: 6fef0de4-89d3-4acb-bfa0-9f67eca9c3f5] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 2e2b4e94-b7e3-44ca-85a2-06de83bce3e0, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 6fef0de4-89d3-4acb-bfa0-9f67eca9c3f5 was re-scheduled: Binding failed for port 2e2b4e94-b7e3-44ca-85a2-06de83bce3e0, please check neutron logs for more information.\n'] [ 820.651298] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 820.651527] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6fef0de4-89d3-4acb-bfa0-9f67eca9c3f5.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6fef0de4-89d3-4acb-bfa0-9f67eca9c3f5. [ 820.651886] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] [instance: 6fef0de4-89d3-4acb-bfa0-9f67eca9c3f5] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6fef0de4-89d3-4acb-bfa0-9f67eca9c3f5. [ 820.681180] nova-conductor[54432]: DEBUG nova.network.neutron [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] [instance: 6fef0de4-89d3-4acb-bfa0-9f67eca9c3f5] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 820.701433] nova-conductor[54432]: DEBUG nova.network.neutron [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] [instance: 6fef0de4-89d3-4acb-bfa0-9f67eca9c3f5] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 820.706704] nova-conductor[54432]: DEBUG nova.network.neutron [None req-cc2a4911-53bc-4563-aae5-9bb09073f3f0 tempest-TenantUsagesTestJSON-463017862 tempest-TenantUsagesTestJSON-463017862-project-member] [instance: 6fef0de4-89d3-4acb-bfa0-9f67eca9c3f5] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 821.674558] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=334,nova_cell1:SAVEPOINT=26,nova_cell1:RELEASE=26,nova_cell1:UPDATE=169,nova_cell1:INSERT=27,nova_cell1:DELETE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 825.238276] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 7bdd4371-b5d5-4053-93b1-75d1c5b9835b] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 1bd118fa-8a80-4b4f-bf10-f9dc71dae0bb, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 7bdd4371-b5d5-4053-93b1-75d1c5b9835b was re-scheduled: Binding failed for port 1bd118fa-8a80-4b4f-bf10-f9dc71dae0bb, please check neutron logs for more information.\n'] [ 825.239210] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 825.239210] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7bdd4371-b5d5-4053-93b1-75d1c5b9835b.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7bdd4371-b5d5-4053-93b1-75d1c5b9835b. [ 825.239838] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 7bdd4371-b5d5-4053-93b1-75d1c5b9835b] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7bdd4371-b5d5-4053-93b1-75d1c5b9835b. [ 825.272103] nova-conductor[54431]: DEBUG nova.network.neutron [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 7bdd4371-b5d5-4053-93b1-75d1c5b9835b] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 825.292247] nova-conductor[54431]: DEBUG nova.network.neutron [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 7bdd4371-b5d5-4053-93b1-75d1c5b9835b] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 825.295771] nova-conductor[54431]: DEBUG nova.network.neutron [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 7bdd4371-b5d5-4053-93b1-75d1c5b9835b] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 825.532684] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 825.546240] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 825.546491] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 825.546660] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 825.576062] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 825.576201] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 825.576372] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 825.576734] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 825.576919] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 825.577099] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 825.584712] nova-conductor[54431]: DEBUG nova.quota [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Getting quotas for project c0f234b7555a473ebc3191c912d57c40. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 825.587127] nova-conductor[54431]: DEBUG nova.quota [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Getting quotas for user f4f4a5ed581245e1a4fd36af472962ef and project c0f234b7555a473ebc3191c912d57c40. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 825.592584] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] [instance: a10b9328-394e-4673-928e-e6fa87b8c844] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 825.592584] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 825.592740] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 825.592897] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 825.595910] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] [instance: a10b9328-394e-4673-928e-e6fa87b8c844] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 825.596535] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 825.596735] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 825.596898] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 825.610664] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 825.610664] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 825.610771] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 827.181895] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 63b76c39-11ca-4f1e-b336-2caceece1f72] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 818ca52d-49e7-4e46-a1ac-50d975693883, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 63b76c39-11ca-4f1e-b336-2caceece1f72 was re-scheduled: Binding failed for port 818ca52d-49e7-4e46-a1ac-50d975693883, please check neutron logs for more information.\n'] [ 827.182826] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 827.183713] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 63b76c39-11ca-4f1e-b336-2caceece1f72.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 63b76c39-11ca-4f1e-b336-2caceece1f72. [ 827.183713] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 63b76c39-11ca-4f1e-b336-2caceece1f72] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 63b76c39-11ca-4f1e-b336-2caceece1f72. [ 827.207789] nova-conductor[54432]: DEBUG nova.network.neutron [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 63b76c39-11ca-4f1e-b336-2caceece1f72] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 827.225485] nova-conductor[54432]: DEBUG nova.network.neutron [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 63b76c39-11ca-4f1e-b336-2caceece1f72] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 827.228648] nova-conductor[54432]: DEBUG nova.network.neutron [None req-ecd0d129-2452-4890-8e73-192545d63492 tempest-MultipleCreateTestJSON-988372643 tempest-MultipleCreateTestJSON-988372643-project-member] [instance: 63b76c39-11ca-4f1e-b336-2caceece1f72] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 829.437008] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 829.451652] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=4,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 833.401208] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 833.417837] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 833.418122] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 833.418387] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 833.455364] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 833.455603] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 833.455763] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 833.456168] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 833.456352] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 833.456505] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 833.466943] nova-conductor[54431]: DEBUG nova.quota [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Getting quotas for project 0accdd542a294475857a3c8b139b536f. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 833.470157] nova-conductor[54431]: DEBUG nova.quota [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Getting quotas for user 2d9df254dd064811923728d85324d67d and project 0accdd542a294475857a3c8b139b536f. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 833.478934] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] [instance: d308417b-47d0-423a-a603-56d9251ef818] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 833.479450] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 833.479677] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 833.480231] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 833.485092] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] [instance: d308417b-47d0-423a-a603-56d9251ef818] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 833.485751] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 833.485973] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 833.486154] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 833.508379] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 833.508583] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 833.508999] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 843.460044] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell0:SELECT=2 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 843.488280] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=10,nova_cell1:UPDATE=3,nova_cell1:INSERT=17,nova_cell1:SAVEPOINT=2,nova_cell1:RELEASE=2 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 843.499612] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=11,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 843.503906] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=15,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 843.548974] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] [instance: 261cbefa-fb2d-48da-a4fe-80b744a931f6] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 1863af9b-2409-4da1-a6dc-d2bb4324e5ec, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 261cbefa-fb2d-48da-a4fe-80b744a931f6 was re-scheduled: Binding failed for port 1863af9b-2409-4da1-a6dc-d2bb4324e5ec, please check neutron logs for more information.\n'] [ 843.548974] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 843.548974] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 261cbefa-fb2d-48da-a4fe-80b744a931f6.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 261cbefa-fb2d-48da-a4fe-80b744a931f6. [ 843.548974] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] [instance: 261cbefa-fb2d-48da-a4fe-80b744a931f6] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 261cbefa-fb2d-48da-a4fe-80b744a931f6. [ 843.569267] nova-conductor[54431]: DEBUG nova.network.neutron [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] [instance: 261cbefa-fb2d-48da-a4fe-80b744a931f6] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 843.590033] nova-conductor[54431]: DEBUG nova.network.neutron [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] [instance: 261cbefa-fb2d-48da-a4fe-80b744a931f6] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 843.593530] nova-conductor[54431]: DEBUG nova.network.neutron [None req-86297533-7bbc-4aeb-be79-5ff5e2437213 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] [instance: 261cbefa-fb2d-48da-a4fe-80b744a931f6] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 858.053362] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] [instance: 49f9108c-7256-4a2c-9ffd-a6d041a180e4] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 80575751-bae4-48a6-b43b-1a4c37d4b3fd, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 49f9108c-7256-4a2c-9ffd-a6d041a180e4 was re-scheduled: Binding failed for port 80575751-bae4-48a6-b43b-1a4c37d4b3fd, please check neutron logs for more information.\n'] [ 858.054021] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 858.054286] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 49f9108c-7256-4a2c-9ffd-a6d041a180e4.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 49f9108c-7256-4a2c-9ffd-a6d041a180e4. [ 858.054540] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] [instance: 49f9108c-7256-4a2c-9ffd-a6d041a180e4] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 49f9108c-7256-4a2c-9ffd-a6d041a180e4. [ 858.086410] nova-conductor[54431]: DEBUG nova.network.neutron [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] [instance: 49f9108c-7256-4a2c-9ffd-a6d041a180e4] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 858.178289] nova-conductor[54431]: DEBUG nova.network.neutron [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] [instance: 49f9108c-7256-4a2c-9ffd-a6d041a180e4] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 858.181343] nova-conductor[54431]: DEBUG nova.network.neutron [None req-bf7ce378-1738-4f88-95f8-a43cfe634856 tempest-ListImageFiltersTestJSON-1745683704 tempest-ListImageFiltersTestJSON-1745683704-project-member] [instance: 49f9108c-7256-4a2c-9ffd-a6d041a180e4] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 858.960745] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 858.973378] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 858.973611] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 858.973779] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 859.010919] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 859.011163] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 859.011336] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 859.011717] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 859.011909] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 859.012083] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 859.021243] nova-conductor[54431]: DEBUG nova.quota [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Getting quotas for project 7ed0ce48db954ddc8ec4461142dd0a6c. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 859.024193] nova-conductor[54431]: DEBUG nova.quota [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Getting quotas for user 162d45fcc74e47f58f1d88f618720cc2 and project 7ed0ce48db954ddc8ec4461142dd0a6c. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 859.029311] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] [instance: b534fcdb-b0db-4e07-85fc-696dc58168aa] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 859.029741] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 859.029969] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 859.030157] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 859.033045] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] [instance: b534fcdb-b0db-4e07-85fc-696dc58168aa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 859.034064] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 859.034064] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 859.034064] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 859.052571] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 859.052813] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 859.052984] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-841b55b1-1ace-4c65-9409-c7753aa92bbb tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 861.774485] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] [instance: d650a1a5-3706-4682-a813-f85ea23098e2] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b80023b9-103c-4a15-948f-4139319d4feb, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance d650a1a5-3706-4682-a813-f85ea23098e2 was re-scheduled: Binding failed for port b80023b9-103c-4a15-948f-4139319d4feb, please check neutron logs for more information.\n'] [ 861.775062] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 861.775301] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d650a1a5-3706-4682-a813-f85ea23098e2.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d650a1a5-3706-4682-a813-f85ea23098e2. [ 861.775534] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] [instance: d650a1a5-3706-4682-a813-f85ea23098e2] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d650a1a5-3706-4682-a813-f85ea23098e2. [ 861.800319] nova-conductor[54432]: DEBUG nova.network.neutron [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] [instance: d650a1a5-3706-4682-a813-f85ea23098e2] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 861.820576] nova-conductor[54432]: DEBUG nova.network.neutron [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] [instance: d650a1a5-3706-4682-a813-f85ea23098e2] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 861.823927] nova-conductor[54432]: DEBUG nova.network.neutron [None req-e48a4632-10ee-406f-b432-282041b25d20 tempest-ServerRescueTestJSONUnderV235-1097481186 tempest-ServerRescueTestJSONUnderV235-1097481186-project-member] [instance: d650a1a5-3706-4682-a813-f85ea23098e2] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 863.551843] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] [instance: 60a4b8c3-9dc4-4867-adca-4503e61ce237] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port d8305010-f9eb-4feb-8f48-828f581c7b97, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 60a4b8c3-9dc4-4867-adca-4503e61ce237 was re-scheduled: Binding failed for port d8305010-f9eb-4feb-8f48-828f581c7b97, please check neutron logs for more information.\n'] [ 863.552464] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 863.552784] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 60a4b8c3-9dc4-4867-adca-4503e61ce237.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 60a4b8c3-9dc4-4867-adca-4503e61ce237. [ 863.553018] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] [instance: 60a4b8c3-9dc4-4867-adca-4503e61ce237] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 60a4b8c3-9dc4-4867-adca-4503e61ce237. [ 863.583754] nova-conductor[54432]: DEBUG nova.network.neutron [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] [instance: 60a4b8c3-9dc4-4867-adca-4503e61ce237] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 863.624041] nova-conductor[54432]: DEBUG nova.network.neutron [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] [instance: 60a4b8c3-9dc4-4867-adca-4503e61ce237] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 863.625045] nova-conductor[54432]: DEBUG nova.network.neutron [None req-1a70708b-ce1e-442f-8855-8a06d7436083 tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] [instance: 60a4b8c3-9dc4-4867-adca-4503e61ce237] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 865.417029] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] [instance: ad749135-08c1-48ca-af56-300d6e796012] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 69123df3-56d1-490c-9303-b6427e432f7f, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ad749135-08c1-48ca-af56-300d6e796012 was re-scheduled: Binding failed for port 69123df3-56d1-490c-9303-b6427e432f7f, please check neutron logs for more information.\n'] [ 865.417711] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 865.418580] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ad749135-08c1-48ca-af56-300d6e796012.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ad749135-08c1-48ca-af56-300d6e796012. [ 865.418580] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] [instance: ad749135-08c1-48ca-af56-300d6e796012] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ad749135-08c1-48ca-af56-300d6e796012. [ 865.443686] nova-conductor[54432]: DEBUG nova.network.neutron [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] [instance: ad749135-08c1-48ca-af56-300d6e796012] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 865.463966] nova-conductor[54432]: DEBUG nova.network.neutron [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] [instance: ad749135-08c1-48ca-af56-300d6e796012] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 865.467961] nova-conductor[54432]: DEBUG nova.network.neutron [None req-c07b55e9-c29b-4c97-a3b8-b6f485ef2cce tempest-ServerRescueNegativeTestJSON-1957428111 tempest-ServerRescueNegativeTestJSON-1957428111-project-member] [instance: ad749135-08c1-48ca-af56-300d6e796012] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 865.717716] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 865.730118] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 865.730356] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 865.730523] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 865.766078] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 865.766078] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 865.766202] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 865.766479] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 865.766640] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 865.766797] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 865.778273] nova-conductor[54432]: DEBUG nova.quota [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Getting quotas for project 616ea9f31e4b4823a1a50b89e3ea4c04. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 865.781095] nova-conductor[54432]: DEBUG nova.quota [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Getting quotas for user 407c9b11e8f74200b1c4e34698925a92 and project 616ea9f31e4b4823a1a50b89e3ea4c04. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 865.785997] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] [instance: f1b5512b-702f-45b1-9844-13a1218c31b3] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 865.786506] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 865.786703] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 865.787312] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 865.790174] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] [instance: f1b5512b-702f-45b1-9844-13a1218c31b3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 865.790800] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 865.791028] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 865.791199] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 865.806223] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 865.806437] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 865.806634] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 867.361475] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] [instance: 2a1c52ab-8ee6-4b63-ad04-2450a60262d0] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b7ead464-8575-4153-96fc-fab89ca560f8, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 2a1c52ab-8ee6-4b63-ad04-2450a60262d0 was re-scheduled: Binding failed for port b7ead464-8575-4153-96fc-fab89ca560f8, please check neutron logs for more information.\n'] [ 867.365211] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 867.365495] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2a1c52ab-8ee6-4b63-ad04-2450a60262d0.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2a1c52ab-8ee6-4b63-ad04-2450a60262d0. [ 867.365957] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] [instance: 2a1c52ab-8ee6-4b63-ad04-2450a60262d0] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2a1c52ab-8ee6-4b63-ad04-2450a60262d0. [ 867.410059] nova-conductor[54431]: DEBUG nova.network.neutron [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] [instance: 2a1c52ab-8ee6-4b63-ad04-2450a60262d0] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 867.428337] nova-conductor[54431]: DEBUG nova.network.neutron [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] [instance: 2a1c52ab-8ee6-4b63-ad04-2450a60262d0] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 867.431499] nova-conductor[54431]: DEBUG nova.network.neutron [None req-708aeb0e-76d9-428b-96c9-50a39a2ef990 tempest-ServersTestFqdnHostnames-1879826867 tempest-ServersTestFqdnHostnames-1879826867-project-member] [instance: 2a1c52ab-8ee6-4b63-ad04-2450a60262d0] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 869.016086] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell0:SELECT=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 869.036299] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=3,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=8,nova_cell1:RELEASE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 869.043580] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=5,nova_api:UPDATE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 869.048815] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=7,nova_api:DELETE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 869.255697] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 6da95995-0527-4b05-94dc-a4fcf0ae2f92] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 392f73a3-4972-4a0b-a13a-fef5efe59658, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 6da95995-0527-4b05-94dc-a4fcf0ae2f92 was re-scheduled: Binding failed for port 392f73a3-4972-4a0b-a13a-fef5efe59658, please check neutron logs for more information.\n'] [ 869.257026] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 869.257205] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6da95995-0527-4b05-94dc-a4fcf0ae2f92.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6da95995-0527-4b05-94dc-a4fcf0ae2f92. [ 869.257658] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 6da95995-0527-4b05-94dc-a4fcf0ae2f92] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6da95995-0527-4b05-94dc-a4fcf0ae2f92. [ 869.285065] nova-conductor[54432]: DEBUG nova.network.neutron [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 6da95995-0527-4b05-94dc-a4fcf0ae2f92] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 869.302631] nova-conductor[54432]: DEBUG nova.network.neutron [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 6da95995-0527-4b05-94dc-a4fcf0ae2f92] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 869.307511] nova-conductor[54432]: DEBUG nova.network.neutron [None req-4641cb07-ea4d-4b99-9ae8-8b9b53720486 tempest-AttachInterfacesTestJSON-1875529461 tempest-AttachInterfacesTestJSON-1875529461-project-member] [instance: 6da95995-0527-4b05-94dc-a4fcf0ae2f92] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 873.544104] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 263931cd-b2dc-41bb-8a2b-abf61aadafc9] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port f0eb55e9-0018-4c0e-99cb-227551afe2c8, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 263931cd-b2dc-41bb-8a2b-abf61aadafc9 was re-scheduled: Binding failed for port f0eb55e9-0018-4c0e-99cb-227551afe2c8, please check neutron logs for more information.\n'] [ 873.544722] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 873.544946] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 263931cd-b2dc-41bb-8a2b-abf61aadafc9.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 263931cd-b2dc-41bb-8a2b-abf61aadafc9. [ 873.547915] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 263931cd-b2dc-41bb-8a2b-abf61aadafc9] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 263931cd-b2dc-41bb-8a2b-abf61aadafc9. [ 873.574458] nova-conductor[54432]: DEBUG nova.network.neutron [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 263931cd-b2dc-41bb-8a2b-abf61aadafc9] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 873.591496] nova-conductor[54432]: DEBUG nova.network.neutron [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 263931cd-b2dc-41bb-8a2b-abf61aadafc9] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 873.594763] nova-conductor[54432]: DEBUG nova.network.neutron [None req-aa5be995-82c9-4e5c-b672-c3cd63dccbe5 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 263931cd-b2dc-41bb-8a2b-abf61aadafc9] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 874.143039] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 874.155888] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 874.156164] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 874.156338] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 874.192839] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 874.192839] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 874.192839] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 874.192839] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 874.192839] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 874.192839] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 874.202881] nova-conductor[54432]: DEBUG nova.quota [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Getting quotas for project 7e4a0adf95ad42cfa20c49eda388b6ce. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 874.206197] nova-conductor[54432]: DEBUG nova.quota [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Getting quotas for user c0026a72212047e48515d4e1d7c43f9b and project 7e4a0adf95ad42cfa20c49eda388b6ce. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 874.211851] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] [instance: 4af6c7f4-b774-4c3f-80af-e8704a904c38] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 874.212100] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 874.212206] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 874.212432] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 874.215227] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] [instance: 4af6c7f4-b774-4c3f-80af-e8704a904c38] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 874.215870] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 874.216081] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 874.216283] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 874.230725] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 874.230988] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 874.231193] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 874.842135] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 874.854629] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 874.854829] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 874.855022] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 874.885675] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 874.885870] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 874.886099] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 874.886422] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 874.886606] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 874.886763] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 874.896628] nova-conductor[54432]: DEBUG nova.quota [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Getting quotas for project 3f11318c300c4c308f3f8a14db355954. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 874.899797] nova-conductor[54432]: DEBUG nova.quota [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Getting quotas for user 4762545dc5e14c16b5cbb1bad27d90a2 and project 3f11318c300c4c308f3f8a14db355954. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 874.906486] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 4fa63411-a721-4e34-a08d-90587b0b34d3] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 874.907159] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 874.907375] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 874.907703] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 874.911414] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 4fa63411-a721-4e34-a08d-90587b0b34d3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 874.912293] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 874.912293] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 874.912547] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 874.946994] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 874.947295] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 874.948430] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 875.328291] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] [instance: 20e4ff1b-0b84-477d-a0d0-a85a439449ad] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 588d004d-9a19-4de8-9e31-22dc0aeede5b, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 20e4ff1b-0b84-477d-a0d0-a85a439449ad was re-scheduled: Binding failed for port 588d004d-9a19-4de8-9e31-22dc0aeede5b, please check neutron logs for more information.\n'] [ 875.328915] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 875.329254] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 20e4ff1b-0b84-477d-a0d0-a85a439449ad.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 20e4ff1b-0b84-477d-a0d0-a85a439449ad. [ 875.330128] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] [instance: 20e4ff1b-0b84-477d-a0d0-a85a439449ad] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 20e4ff1b-0b84-477d-a0d0-a85a439449ad. [ 875.355676] nova-conductor[54431]: DEBUG nova.network.neutron [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] [instance: 20e4ff1b-0b84-477d-a0d0-a85a439449ad] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 875.372745] nova-conductor[54431]: DEBUG nova.network.neutron [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] [instance: 20e4ff1b-0b84-477d-a0d0-a85a439449ad] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 875.375836] nova-conductor[54431]: DEBUG nova.network.neutron [None req-d5fffe0e-8595-4212-b847-9f4fb17f5bc9 tempest-ServersNegativeTestMultiTenantJSON-1317981700 tempest-ServersNegativeTestMultiTenantJSON-1317981700-project-member] [instance: 20e4ff1b-0b84-477d-a0d0-a85a439449ad] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 876.235771] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=410,nova_cell1:UPDATE=224,nova_cell1:INSERT=29,nova_cell1:SAVEPOINT=42,nova_cell1:RELEASE=42,nova_cell1:DELETE=4 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 876.623381] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 876.639180] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 876.641761] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 876.641761] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 876.670457] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 876.670681] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 876.670856] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 876.671317] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 876.671382] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 876.671603] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 876.678971] nova-conductor[54431]: DEBUG nova.quota [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Getting quotas for project fcf8aa231c564984b42197ad6a93e018. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 876.681154] nova-conductor[54431]: DEBUG nova.quota [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Getting quotas for user f4fd324459ba49f7b40aad54e650d925 and project fcf8aa231c564984b42197ad6a93e018. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 876.686364] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] [instance: 88466440-7392-43d3-83ec-45229516a3ef] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 876.686793] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 876.686992] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 876.687175] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 876.691360] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] [instance: 88466440-7392-43d3-83ec-45229516a3ef] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 876.691360] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 876.691631] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 876.691631] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 876.708826] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 876.709076] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 876.709256] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 877.184947] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] [instance: 4eda80db-5db7-4fd6-9932-108c22f4a616] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port dee6d343-e56a-4d3a-9f5f-2a6fe259825e, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 4eda80db-5db7-4fd6-9932-108c22f4a616 was re-scheduled: Binding failed for port dee6d343-e56a-4d3a-9f5f-2a6fe259825e, please check neutron logs for more information.\n'] [ 877.185800] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 877.187548] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4eda80db-5db7-4fd6-9932-108c22f4a616.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4eda80db-5db7-4fd6-9932-108c22f4a616. [ 877.187548] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] [instance: 4eda80db-5db7-4fd6-9932-108c22f4a616] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4eda80db-5db7-4fd6-9932-108c22f4a616. [ 877.214459] nova-conductor[54432]: DEBUG nova.network.neutron [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] [instance: 4eda80db-5db7-4fd6-9932-108c22f4a616] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 877.233775] nova-conductor[54432]: DEBUG nova.network.neutron [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] [instance: 4eda80db-5db7-4fd6-9932-108c22f4a616] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 877.236891] nova-conductor[54432]: DEBUG nova.network.neutron [None req-3bd4fa14-cb89-4456-85a7-187db1076363 tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] [instance: 4eda80db-5db7-4fd6-9932-108c22f4a616] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 880.677635] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 880.691605] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 880.691851] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 880.692038] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 880.733606] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 880.734446] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 880.734446] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 880.734446] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 880.734674] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 880.734733] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 880.745258] nova-conductor[54432]: DEBUG nova.quota [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Getting quotas for project 967135c837004321824783716458da8e. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 880.748314] nova-conductor[54432]: DEBUG nova.quota [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Getting quotas for user ffc458db78a04358af903d6119c5eac9 and project 967135c837004321824783716458da8e. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 880.753976] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] [instance: 95a5cb2c-24da-43c0-b9d1-7f0b51ccda87] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 880.754567] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 880.755639] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 880.755639] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 880.760750] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] [instance: 95a5cb2c-24da-43c0-b9d1-7f0b51ccda87] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 880.761426] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 880.761628] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 880.761791] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 880.778097] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 880.778314] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 880.778479] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 881.926681] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=392,nova_cell1:UPDATE=217,nova_cell1:SAVEPOINT=37,nova_cell1:RELEASE=37,nova_cell1:INSERT=29,nova_cell1:DELETE=4 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 886.675076] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell0:SELECT=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 886.693592] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=3,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=8,nova_cell1:RELEASE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 886.704841] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=7 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 886.706896] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=5,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 890.738143] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell0:SELECT=4 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 890.764423] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=12,nova_cell1:SAVEPOINT=2,nova_cell1:INSERT=30,nova_cell1:RELEASE=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 890.773402] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=27,nova_api:DELETE=2,nova_api:UPDATE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 890.775255] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=21,nova_api:UPDATE=3,nova_api:DELETE=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 894.586849] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: c1961f60-06d8-44d8-999f-d498c3667adc] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 296e37b0-c4ab-4591-9b5f-d51c80bb0f9c, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance c1961f60-06d8-44d8-999f-d498c3667adc was re-scheduled: Binding failed for port 296e37b0-c4ab-4591-9b5f-d51c80bb0f9c, please check neutron logs for more information.\n'] [ 894.587445] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 894.587671] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c1961f60-06d8-44d8-999f-d498c3667adc.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c1961f60-06d8-44d8-999f-d498c3667adc. [ 894.587985] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: c1961f60-06d8-44d8-999f-d498c3667adc] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c1961f60-06d8-44d8-999f-d498c3667adc. [ 894.623553] nova-conductor[54432]: DEBUG nova.network.neutron [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: c1961f60-06d8-44d8-999f-d498c3667adc] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 894.774450] nova-conductor[54432]: DEBUG nova.network.neutron [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: c1961f60-06d8-44d8-999f-d498c3667adc] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 894.777590] nova-conductor[54432]: DEBUG nova.network.neutron [None req-d67bf495-4ce4-44a5-8cd3-fd105e76c45d tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: c1961f60-06d8-44d8-999f-d498c3667adc] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 909.910950] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: ba8fc3e0-dae8-4910-858b-dab1de72089e] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 2a8fbacb-6166-43d0-b9f4-1c05df641aad, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ba8fc3e0-dae8-4910-858b-dab1de72089e was re-scheduled: Binding failed for port 2a8fbacb-6166-43d0-b9f4-1c05df641aad, please check neutron logs for more information.\n'] [ 909.911524] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 909.911738] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ba8fc3e0-dae8-4910-858b-dab1de72089e.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ba8fc3e0-dae8-4910-858b-dab1de72089e. [ 909.911977] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: ba8fc3e0-dae8-4910-858b-dab1de72089e] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ba8fc3e0-dae8-4910-858b-dab1de72089e. [ 909.935103] nova-conductor[54432]: DEBUG nova.network.neutron [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: ba8fc3e0-dae8-4910-858b-dab1de72089e] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 909.951031] nova-conductor[54432]: DEBUG nova.network.neutron [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: ba8fc3e0-dae8-4910-858b-dab1de72089e] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 909.953921] nova-conductor[54432]: DEBUG nova.network.neutron [None req-d99fa83e-f044-4de4-a72d-022ee6946c23 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: ba8fc3e0-dae8-4910-858b-dab1de72089e] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 911.648726] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: 0eaadb76-e2c7-4e06-b865-065ab0f014ba] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 4681981e-8791-4c18-b0e8-49b24f59baa0, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 0eaadb76-e2c7-4e06-b865-065ab0f014ba was re-scheduled: Binding failed for port 4681981e-8791-4c18-b0e8-49b24f59baa0, please check neutron logs for more information.\n'] [ 911.649417] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 911.649711] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0eaadb76-e2c7-4e06-b865-065ab0f014ba.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0eaadb76-e2c7-4e06-b865-065ab0f014ba. [ 911.650178] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: 0eaadb76-e2c7-4e06-b865-065ab0f014ba] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0eaadb76-e2c7-4e06-b865-065ab0f014ba. [ 911.669369] nova-conductor[54431]: DEBUG nova.network.neutron [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: 0eaadb76-e2c7-4e06-b865-065ab0f014ba] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 911.686527] nova-conductor[54431]: DEBUG nova.network.neutron [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: 0eaadb76-e2c7-4e06-b865-065ab0f014ba] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 911.689863] nova-conductor[54431]: DEBUG nova.network.neutron [None req-c09994b6-9b80-4010-907a-2b3984b40ed2 tempest-ListServerFiltersTestJSON-1553140998 tempest-ListServerFiltersTestJSON-1553140998-project-member] [instance: 0eaadb76-e2c7-4e06-b865-065ab0f014ba] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 912.495492] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-dd59643a-3941-445d-89d8-7097abf1cd62 tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Targeting cell 8fb5f300-ab58-4d7c-afe8-8d0b4be88dee(cell1) for conductor method rebuild_instance {{(pid=54432) wrapper /opt/stack/nova/nova/conductor/manager.py:95}} [ 912.495803] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dd59643a-3941-445d-89d8-7097abf1cd62 tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 912.496116] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dd59643a-3941-445d-89d8-7097abf1cd62 tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 912.496370] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-dd59643a-3941-445d-89d8-7097abf1cd62 tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 912.505561] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-dd59643a-3941-445d-89d8-7097abf1cd62 tempest-ServerShowV257Test-1145004636 tempest-ServerShowV257Test-1145004636-project-member] [instance: b534fcdb-b0db-4e07-85fc-696dc58168aa] No migration record for the rebuild/evacuate request. {{(pid=54432) rebuild_instance /opt/stack/nova/nova/conductor/manager.py:1227}} [ 913.438993] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 764ef9ab-22ae-45cd-8b5d-76a63346d93c] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 9a754f05-ea5f-4418-aa84-0078099e5dce, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 764ef9ab-22ae-45cd-8b5d-76a63346d93c was re-scheduled: Binding failed for port 9a754f05-ea5f-4418-aa84-0078099e5dce, please check neutron logs for more information.\n'] [ 913.440586] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 913.440586] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 764ef9ab-22ae-45cd-8b5d-76a63346d93c.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 764ef9ab-22ae-45cd-8b5d-76a63346d93c. [ 913.440722] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 764ef9ab-22ae-45cd-8b5d-76a63346d93c] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 764ef9ab-22ae-45cd-8b5d-76a63346d93c. [ 913.463488] nova-conductor[54431]: DEBUG nova.network.neutron [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 764ef9ab-22ae-45cd-8b5d-76a63346d93c] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 913.478598] nova-conductor[54431]: DEBUG nova.network.neutron [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 764ef9ab-22ae-45cd-8b5d-76a63346d93c] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 913.481727] nova-conductor[54431]: DEBUG nova.network.neutron [None req-b73f9761-6eed-4f72-b930-96fb390553b6 tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 764ef9ab-22ae-45cd-8b5d-76a63346d93c] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 914.717129] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 914.729389] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 914.729622] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 914.729787] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 914.773934] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 914.774165] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 914.774335] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 914.774687] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 914.774869] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 914.775041] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 914.782355] nova-conductor[54431]: DEBUG nova.quota [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Getting quotas for project e444950a637b4fdab2edb1d62aac4be5. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 914.784550] nova-conductor[54431]: DEBUG nova.quota [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Getting quotas for user c92b9d37f796479ca3b6d9f0ed1789c8 and project e444950a637b4fdab2edb1d62aac4be5. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 914.790024] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 9fd7db7e-466e-4213-9e1d-210eae714884] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 914.790254] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 914.790450] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 914.790612] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 914.793881] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 9fd7db7e-466e-4213-9e1d-210eae714884] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 914.794503] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 914.794697] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 914.794858] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 914.807879] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 914.808154] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 914.808344] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 915.264167] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: a92509e3-d5b1-41d3-b9af-fa0af8a956c9] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 0349f93a-0a30-49e4-a1c6-0c72ee6a71c9, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance a92509e3-d5b1-41d3-b9af-fa0af8a956c9 was re-scheduled: Binding failed for port 0349f93a-0a30-49e4-a1c6-0c72ee6a71c9, please check neutron logs for more information.\n'] [ 915.264817] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 915.265096] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a92509e3-d5b1-41d3-b9af-fa0af8a956c9.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a92509e3-d5b1-41d3-b9af-fa0af8a956c9. [ 915.265483] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: a92509e3-d5b1-41d3-b9af-fa0af8a956c9] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a92509e3-d5b1-41d3-b9af-fa0af8a956c9. [ 915.286669] nova-conductor[54431]: DEBUG nova.network.neutron [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: a92509e3-d5b1-41d3-b9af-fa0af8a956c9] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 915.305735] nova-conductor[54431]: DEBUG nova.network.neutron [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: a92509e3-d5b1-41d3-b9af-fa0af8a956c9] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 915.308159] nova-conductor[54431]: DEBUG nova.network.neutron [None req-ebf251d1-e02e-4f79-b4f9-ee2ca970b777 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: a92509e3-d5b1-41d3-b9af-fa0af8a956c9] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 916.563244] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 916.577732] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 916.577732] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 916.577913] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 916.603826] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 916.604062] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 916.604235] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 916.604592] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 916.604771] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 916.604926] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 916.612723] nova-conductor[54431]: DEBUG nova.quota [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Getting quotas for project c9f95d7e954f4413b01aad519c498967. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 916.614823] nova-conductor[54431]: DEBUG nova.quota [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Getting quotas for user 7da3078e8bb5444c96e007cfc2d57952 and project c9f95d7e954f4413b01aad519c498967. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 916.619702] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: ea4b6c62-2880-4a59-8017-b5d1518c1060] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 916.620140] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 916.620350] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 916.620513] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 916.625409] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: ea4b6c62-2880-4a59-8017-b5d1518c1060] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 916.626038] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 916.626242] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 916.626405] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 916.639230] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 916.639482] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 916.639684] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 916.949586] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 770160d6-4681-40e0-a812-5c491b804da4] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 982d43f5-d2d0-473c-a37b-1cb4ec42ab11, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 770160d6-4681-40e0-a812-5c491b804da4 was re-scheduled: Binding failed for port 982d43f5-d2d0-473c-a37b-1cb4ec42ab11, please check neutron logs for more information.\n'] [ 916.950151] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 916.950392] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 770160d6-4681-40e0-a812-5c491b804da4.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 770160d6-4681-40e0-a812-5c491b804da4. [ 916.950699] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 770160d6-4681-40e0-a812-5c491b804da4] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 770160d6-4681-40e0-a812-5c491b804da4. [ 916.975746] nova-conductor[54431]: DEBUG nova.network.neutron [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 770160d6-4681-40e0-a812-5c491b804da4] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 916.991789] nova-conductor[54431]: DEBUG nova.network.neutron [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 770160d6-4681-40e0-a812-5c491b804da4] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 916.994652] nova-conductor[54431]: DEBUG nova.network.neutron [None req-7aeb3b99-c914-4c2d-8ca6-8bdb31a653d6 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 770160d6-4681-40e0-a812-5c491b804da4] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 918.704143] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] [instance: cbe51201-c036-4f90-9372-8cc2d033f7a2] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 765be7c7-1b25-4959-b478-d5f415357706, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance cbe51201-c036-4f90-9372-8cc2d033f7a2 was re-scheduled: Binding failed for port 765be7c7-1b25-4959-b478-d5f415357706, please check neutron logs for more information.\n'] [ 918.704966] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 918.705290] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance cbe51201-c036-4f90-9372-8cc2d033f7a2.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance cbe51201-c036-4f90-9372-8cc2d033f7a2. [ 918.705591] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] [instance: cbe51201-c036-4f90-9372-8cc2d033f7a2] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance cbe51201-c036-4f90-9372-8cc2d033f7a2. [ 918.733598] nova-conductor[54431]: DEBUG nova.network.neutron [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] [instance: cbe51201-c036-4f90-9372-8cc2d033f7a2] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 918.748169] nova-conductor[54431]: DEBUG nova.network.neutron [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] [instance: cbe51201-c036-4f90-9372-8cc2d033f7a2] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 918.751695] nova-conductor[54431]: DEBUG nova.network.neutron [None req-22b68b03-6b51-450c-905b-0fcd89def90e tempest-ServerActionsTestJSON-711130886 tempest-ServerActionsTestJSON-711130886-project-member] [instance: cbe51201-c036-4f90-9372-8cc2d033f7a2] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 920.149679] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 920.163759] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 920.163990] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 920.164181] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 920.192610] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 920.192967] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 920.193175] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 920.193572] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 920.193736] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 920.193898] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 920.221236] nova-conductor[54432]: DEBUG nova.quota [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Getting quotas for project cb9ca6d8c8ae4352bc3c7b1daa7ba288. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 920.237290] nova-conductor[54432]: DEBUG nova.quota [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Getting quotas for user 88c76f25140f4de28bd719de1d5ff616 and project cb9ca6d8c8ae4352bc3c7b1daa7ba288. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 920.246810] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 99920efd-13e6-4a18-9532-4b53bea3c736] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 920.247477] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 920.247729] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 920.247912] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 920.256727] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 99920efd-13e6-4a18-9532-4b53bea3c736] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 920.257618] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 920.257862] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 920.258066] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 920.314938] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 920.315211] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 920.315359] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 922.698560] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] [instance: a10b9328-394e-4673-928e-e6fa87b8c844] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 91890880-66f0-40a4-a58f-6ad5b358a57a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance a10b9328-394e-4673-928e-e6fa87b8c844 was re-scheduled: Binding failed for port 91890880-66f0-40a4-a58f-6ad5b358a57a, please check neutron logs for more information.\n'] [ 922.699272] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 922.699537] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a10b9328-394e-4673-928e-e6fa87b8c844.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a10b9328-394e-4673-928e-e6fa87b8c844. [ 922.700087] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] [instance: a10b9328-394e-4673-928e-e6fa87b8c844] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a10b9328-394e-4673-928e-e6fa87b8c844. [ 922.727037] nova-conductor[54432]: DEBUG nova.network.neutron [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] [instance: a10b9328-394e-4673-928e-e6fa87b8c844] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 922.742530] nova-conductor[54432]: DEBUG nova.network.neutron [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] [instance: a10b9328-394e-4673-928e-e6fa87b8c844] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 922.745156] nova-conductor[54432]: DEBUG nova.network.neutron [None req-610ab5d6-e011-4a99-af58-edb9cdf6804b tempest-ServerAddressesNegativeTestJSON-1867545015 tempest-ServerAddressesNegativeTestJSON-1867545015-project-member] [instance: a10b9328-394e-4673-928e-e6fa87b8c844] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 924.508792] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] [instance: d308417b-47d0-423a-a603-56d9251ef818] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port aa3bd8c0-68d7-4a18-86d7-24b006731bea, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance d308417b-47d0-423a-a603-56d9251ef818 was re-scheduled: Binding failed for port aa3bd8c0-68d7-4a18-86d7-24b006731bea, please check neutron logs for more information.\n'] [ 924.509264] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 924.509489] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d308417b-47d0-423a-a603-56d9251ef818.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d308417b-47d0-423a-a603-56d9251ef818. [ 924.509692] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] [instance: d308417b-47d0-423a-a603-56d9251ef818] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d308417b-47d0-423a-a603-56d9251ef818. [ 924.550427] nova-conductor[54432]: DEBUG nova.network.neutron [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] [instance: d308417b-47d0-423a-a603-56d9251ef818] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 924.575659] nova-conductor[54432]: DEBUG nova.network.neutron [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] [instance: d308417b-47d0-423a-a603-56d9251ef818] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 924.578800] nova-conductor[54432]: DEBUG nova.network.neutron [None req-2cfec637-10c0-4507-8b23-d16501f8ae53 tempest-ServerAddressesTestJSON-249145922 tempest-ServerAddressesTestJSON-249145922-project-member] [instance: d308417b-47d0-423a-a603-56d9251ef818] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 926.609988] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell0:SELECT=2 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 926.629195] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=6,nova_cell1:INSERT=14 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 926.636166] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=14 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 926.638123] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=10,nova_api:UPDATE=2,nova_api:DELETE=2 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 929.350593] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 929.365505] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 929.365741] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 929.365909] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 929.400607] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 929.401136] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 929.401136] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 929.401409] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 929.401598] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 929.401748] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 929.413519] nova-conductor[54432]: DEBUG nova.quota [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Getting quotas for project a7226f416d264f4e8888c523883bd131. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 929.416569] nova-conductor[54432]: DEBUG nova.quota [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Getting quotas for user 07524734e344412e9ddc0904ac1e135f and project a7226f416d264f4e8888c523883bd131. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 929.422683] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] [instance: 4170a2f4-73ff-4bc6-9dc6-d2c4765c449d] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 929.423214] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 929.423481] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 929.423653] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 929.428932] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] [instance: 4170a2f4-73ff-4bc6-9dc6-d2c4765c449d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 929.429599] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 929.429800] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 929.429995] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 929.447331] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 929.447560] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 929.447729] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 931.347097] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 931.369505] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 931.369973] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 931.370355] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 931.415028] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 931.415154] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 931.415318] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 931.415672] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 931.415854] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 931.416018] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 931.425768] nova-conductor[54432]: DEBUG nova.quota [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Getting quotas for project f509dfe537d441bcacb1a4aab3607214. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 931.428617] nova-conductor[54432]: DEBUG nova.quota [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Getting quotas for user 02640299eb9c4a65bc80de6242dd2cd5 and project f509dfe537d441bcacb1a4aab3607214. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 931.436048] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: ca937041-1aa6-46d8-be4e-b4983652fe15] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 931.436485] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 931.436723] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 931.436900] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 931.444015] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: ca937041-1aa6-46d8-be4e-b4983652fe15] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 931.444772] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 931.444843] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 931.445859] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 931.470968] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 931.471103] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 931.472032] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 936.397540] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:UPDATE=176,nova_cell1:SELECT=367,nova_cell1:INSERT=21,nova_cell1:SAVEPOINT=36,nova_cell1:RELEASE=36 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 941.420599] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell0:SELECT=3 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 941.446811] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=13,nova_cell1:UPDATE=3,nova_cell1:INSERT=24,nova_cell1:SAVEPOINT=2,nova_cell1:RELEASE=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 941.456220] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=19,nova_api:UPDATE=1,nova_api:DELETE=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 941.460750] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=19,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 942.061587] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=364,nova_cell1:UPDATE=175,nova_cell1:SAVEPOINT=38,nova_cell1:RELEASE=38,nova_cell1:INSERT=25 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 942.984937] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] [instance: f1b5512b-702f-45b1-9844-13a1218c31b3] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b2333580-9163-4b2c-bd7f-1a2225de00b4, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance f1b5512b-702f-45b1-9844-13a1218c31b3 was re-scheduled: Binding failed for port b2333580-9163-4b2c-bd7f-1a2225de00b4, please check neutron logs for more information.\n'] [ 942.985526] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 942.985753] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f1b5512b-702f-45b1-9844-13a1218c31b3.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f1b5512b-702f-45b1-9844-13a1218c31b3. [ 942.985963] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] [instance: f1b5512b-702f-45b1-9844-13a1218c31b3] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f1b5512b-702f-45b1-9844-13a1218c31b3. [ 943.010558] nova-conductor[54431]: DEBUG nova.network.neutron [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] [instance: f1b5512b-702f-45b1-9844-13a1218c31b3] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 943.025972] nova-conductor[54431]: DEBUG nova.network.neutron [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] [instance: f1b5512b-702f-45b1-9844-13a1218c31b3] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 943.028983] nova-conductor[54431]: DEBUG nova.network.neutron [None req-1ee7959d-e417-4788-8ce5-5e53e9740263 tempest-AttachInterfacesUnderV243Test-220457506 tempest-AttachInterfacesUnderV243Test-220457506-project-member] [instance: f1b5512b-702f-45b1-9844-13a1218c31b3] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 946.594789] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] [instance: 4af6c7f4-b774-4c3f-80af-e8704a904c38] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 21aa756e-d6a2-47b0-92ea-2b12642e43a1, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 4af6c7f4-b774-4c3f-80af-e8704a904c38 was re-scheduled: Binding failed for port 21aa756e-d6a2-47b0-92ea-2b12642e43a1, please check neutron logs for more information.\n'] [ 946.595559] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 946.595559] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4af6c7f4-b774-4c3f-80af-e8704a904c38.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4af6c7f4-b774-4c3f-80af-e8704a904c38. [ 946.596161] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] [instance: 4af6c7f4-b774-4c3f-80af-e8704a904c38] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4af6c7f4-b774-4c3f-80af-e8704a904c38. [ 946.620959] nova-conductor[54432]: DEBUG nova.network.neutron [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] [instance: 4af6c7f4-b774-4c3f-80af-e8704a904c38] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 946.639347] nova-conductor[54432]: DEBUG nova.network.neutron [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] [instance: 4af6c7f4-b774-4c3f-80af-e8704a904c38] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 946.642732] nova-conductor[54432]: DEBUG nova.network.neutron [None req-91d05791-7744-4fb6-b96a-b6945b71df02 tempest-InstanceActionsV221TestJSON-464989833 tempest-InstanceActionsV221TestJSON-464989833-project-member] [instance: 4af6c7f4-b774-4c3f-80af-e8704a904c38] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 950.543867] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 4fa63411-a721-4e34-a08d-90587b0b34d3] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 124, in wait\n result = hub.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch\n return self.greenlet.switch()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port d7f7e92e-10d8-46bd-8a94-a1d88ba5bbdd, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 4fa63411-a721-4e34-a08d-90587b0b34d3 was re-scheduled: Binding failed for port d7f7e92e-10d8-46bd-8a94-a1d88ba5bbdd, please check neutron logs for more information.\n'] [ 950.544484] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 950.544710] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4fa63411-a721-4e34-a08d-90587b0b34d3.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4fa63411-a721-4e34-a08d-90587b0b34d3. [ 950.544949] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 4fa63411-a721-4e34-a08d-90587b0b34d3] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4fa63411-a721-4e34-a08d-90587b0b34d3. [ 950.568145] nova-conductor[54431]: DEBUG nova.network.neutron [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 4fa63411-a721-4e34-a08d-90587b0b34d3] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 950.583964] nova-conductor[54431]: DEBUG nova.network.neutron [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 4fa63411-a721-4e34-a08d-90587b0b34d3] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 950.586639] nova-conductor[54431]: DEBUG nova.network.neutron [None req-184b2e2f-40aa-49c1-a157-298cc02d5fbc tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: 4fa63411-a721-4e34-a08d-90587b0b34d3] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 951.968849] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 951.971634] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 951.991374] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 951.991595] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 951.991756] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 951.993192] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 951.993411] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 951.993571] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 952.035551] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 952.035790] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 952.035960] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 952.036357] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 952.036544] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 952.036701] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 952.040554] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 952.040864] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 952.041135] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 952.041434] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 952.041618] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 952.041772] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 952.046767] nova-conductor[54432]: DEBUG nova.quota [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Getting quotas for project 3f11318c300c4c308f3f8a14db355954. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 952.049963] nova-conductor[54432]: DEBUG nova.quota [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Getting quotas for user 4762545dc5e14c16b5cbb1bad27d90a2 and project 3f11318c300c4c308f3f8a14db355954. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 952.050708] nova-conductor[54431]: DEBUG nova.quota [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Getting quotas for project 1998d4f886c848e1a219e14fa00ef209. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 952.052876] nova-conductor[54431]: DEBUG nova.quota [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Getting quotas for user 10ba90e97f614131adf79a9a8480d702 and project 1998d4f886c848e1a219e14fa00ef209. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 952.055698] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: b4e185e6-b177-4212-bad8-b9fd1b51ea95] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 952.055698] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 952.055827] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 952.055933] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 952.060095] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] [instance: 636d27e5-f184-4656-9225-09c6ec0f0772] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 952.060532] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 952.060840] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: b4e185e6-b177-4212-bad8-b9fd1b51ea95] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 952.061011] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 952.061208] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 952.062367] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 952.062367] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 952.062367] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 952.063608] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] [instance: 636d27e5-f184-4656-9225-09c6ec0f0772] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 952.064348] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 952.064499] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 952.064612] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 952.079973] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 952.080238] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 952.080407] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 952.080619] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 952.080836] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 952.081009] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 952.163504] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] [instance: 88466440-7392-43d3-83ec-45229516a3ef] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 17a49625-1200-4a66-81d3-8a6e144d9234, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 88466440-7392-43d3-83ec-45229516a3ef was re-scheduled: Binding failed for port 17a49625-1200-4a66-81d3-8a6e144d9234, please check neutron logs for more information.\n'] [ 952.164163] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 952.164342] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 88466440-7392-43d3-83ec-45229516a3ef.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 88466440-7392-43d3-83ec-45229516a3ef. [ 952.164652] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] [instance: 88466440-7392-43d3-83ec-45229516a3ef] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 88466440-7392-43d3-83ec-45229516a3ef. [ 952.189761] nova-conductor[54431]: DEBUG nova.network.neutron [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] [instance: 88466440-7392-43d3-83ec-45229516a3ef] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 952.205151] nova-conductor[54431]: DEBUG nova.network.neutron [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] [instance: 88466440-7392-43d3-83ec-45229516a3ef] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 952.208563] nova-conductor[54431]: DEBUG nova.network.neutron [None req-12f3fb3d-058f-4625-8df5-6b9ab03b1e0c tempest-ServerMetadataNegativeTestJSON-1679020876 tempest-ServerMetadataNegativeTestJSON-1679020876-project-member] [instance: 88466440-7392-43d3-83ec-45229516a3ef] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 956.122644] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] [instance: 95a5cb2c-24da-43c0-b9d1-7f0b51ccda87] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port d984b948-6a63-4e30-b2af-b83378f4d62b, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 95a5cb2c-24da-43c0-b9d1-7f0b51ccda87 was re-scheduled: Binding failed for port d984b948-6a63-4e30-b2af-b83378f4d62b, please check neutron logs for more information.\n'] [ 956.123206] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 956.123442] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 95a5cb2c-24da-43c0-b9d1-7f0b51ccda87.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 95a5cb2c-24da-43c0-b9d1-7f0b51ccda87. [ 956.123720] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] [instance: 95a5cb2c-24da-43c0-b9d1-7f0b51ccda87] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 95a5cb2c-24da-43c0-b9d1-7f0b51ccda87. [ 956.146718] nova-conductor[54431]: DEBUG nova.network.neutron [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] [instance: 95a5cb2c-24da-43c0-b9d1-7f0b51ccda87] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 956.165503] nova-conductor[54431]: DEBUG nova.network.neutron [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] [instance: 95a5cb2c-24da-43c0-b9d1-7f0b51ccda87] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 956.169197] nova-conductor[54431]: DEBUG nova.network.neutron [None req-499ee740-6bfe-4145-8bdf-393a38067bde tempest-AttachVolumeTestJSON-287116629 tempest-AttachVolumeTestJSON-287116629-project-member] [instance: 95a5cb2c-24da-43c0-b9d1-7f0b51ccda87] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 958.323020] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 9fd7db7e-466e-4213-9e1d-210eae714884] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port e5633ef6-c4da-4c0c-bf53-6f3e9160d31a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 9fd7db7e-466e-4213-9e1d-210eae714884 was re-scheduled: Binding failed for port e5633ef6-c4da-4c0c-bf53-6f3e9160d31a, please check neutron logs for more information.\n'] [ 958.323020] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 958.323020] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9fd7db7e-466e-4213-9e1d-210eae714884.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9fd7db7e-466e-4213-9e1d-210eae714884. [ 958.323020] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 9fd7db7e-466e-4213-9e1d-210eae714884] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9fd7db7e-466e-4213-9e1d-210eae714884. [ 958.350346] nova-conductor[54431]: DEBUG nova.network.neutron [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 9fd7db7e-466e-4213-9e1d-210eae714884] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 958.372126] nova-conductor[54431]: DEBUG nova.network.neutron [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 9fd7db7e-466e-4213-9e1d-210eae714884] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 958.374380] nova-conductor[54431]: DEBUG nova.network.neutron [None req-53333dec-ce18-43e3-8a0d-bfed00c4be7b tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 9fd7db7e-466e-4213-9e1d-210eae714884] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 958.535393] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 958.558342] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 958.558605] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 958.558774] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 958.595885] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 958.596128] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 958.596311] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 958.596706] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 958.596888] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 958.597053] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 958.607686] nova-conductor[54431]: DEBUG nova.quota [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Getting quotas for project c21dbc475b0d4225b920fbad495b785a. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 958.610576] nova-conductor[54431]: DEBUG nova.quota [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Getting quotas for user 48b2a1fbea0e4e22bfa32ec31e438f6d and project c21dbc475b0d4225b920fbad495b785a. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 958.615622] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] [instance: d9151031-e57a-4bc9-9f6f-ce2d8e74a001] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 958.616066] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 958.616272] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 958.616433] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 958.619372] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] [instance: d9151031-e57a-4bc9-9f6f-ce2d8e74a001] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 958.620122] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 958.620269] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 958.620467] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 958.636449] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 958.636969] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 958.636969] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 959.864090] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: ea4b6c62-2880-4a59-8017-b5d1518c1060] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 4464326a-eda6-4130-9df6-a9d552a560c1, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ea4b6c62-2880-4a59-8017-b5d1518c1060 was re-scheduled: Binding failed for port 4464326a-eda6-4130-9df6-a9d552a560c1, please check neutron logs for more information.\n'] [ 959.864722] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 959.864959] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ea4b6c62-2880-4a59-8017-b5d1518c1060.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ea4b6c62-2880-4a59-8017-b5d1518c1060. [ 959.865283] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: ea4b6c62-2880-4a59-8017-b5d1518c1060] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ea4b6c62-2880-4a59-8017-b5d1518c1060. [ 959.893329] nova-conductor[54431]: DEBUG nova.network.neutron [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: ea4b6c62-2880-4a59-8017-b5d1518c1060] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 959.911314] nova-conductor[54431]: DEBUG nova.network.neutron [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: ea4b6c62-2880-4a59-8017-b5d1518c1060] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 959.915102] nova-conductor[54431]: DEBUG nova.network.neutron [None req-ac9ccd3d-784a-4679-b4a1-84f402696a69 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: ea4b6c62-2880-4a59-8017-b5d1518c1060] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 960.325497] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 960.338381] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 960.338494] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 960.338650] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 960.368532] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 960.368759] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 960.368922] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 960.369332] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 960.369527] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 960.369685] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 960.380171] nova-conductor[54432]: DEBUG nova.quota [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Getting quotas for project e444950a637b4fdab2edb1d62aac4be5. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 960.383329] nova-conductor[54432]: DEBUG nova.quota [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Getting quotas for user c92b9d37f796479ca3b6d9f0ed1789c8 and project e444950a637b4fdab2edb1d62aac4be5. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 960.389068] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 1c1379c2-348b-4be8-9415-594ce53ebf55] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 960.389527] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 960.389919] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 960.389919] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 960.393050] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 1c1379c2-348b-4be8-9415-594ce53ebf55] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 960.393195] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 960.393329] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 960.393494] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 960.407800] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 960.407913] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 960.408099] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 961.077064] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-640f5a23-9c80-44b2-bb1c-3d22fef1d53e tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 961.084781] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-640f5a23-9c80-44b2-bb1c-3d22fef1d53e tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 8e25a1ed-5ed6-47b8-8ea5-c25240c3d493] While scheduling instance, the build request was already deleted. {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1706}} [ 961.204027] nova-conductor[54431]: INFO nova.scheduler.client.report [None req-640f5a23-9c80-44b2-bb1c-3d22fef1d53e tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Deleted allocations for instance 8e25a1ed-5ed6-47b8-8ea5-c25240c3d493 [ 961.209077] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-640f5a23-9c80-44b2-bb1c-3d22fef1d53e tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 961.209293] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-640f5a23-9c80-44b2-bb1c-3d22fef1d53e tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 961.209458] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-640f5a23-9c80-44b2-bb1c-3d22fef1d53e tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 961.209803] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-640f5a23-9c80-44b2-bb1c-3d22fef1d53e tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 961.210044] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-640f5a23-9c80-44b2-bb1c-3d22fef1d53e tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 961.210215] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-640f5a23-9c80-44b2-bb1c-3d22fef1d53e tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 961.218924] nova-conductor[54431]: DEBUG nova.quota [None req-640f5a23-9c80-44b2-bb1c-3d22fef1d53e tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Getting quotas for project c9f95d7e954f4413b01aad519c498967. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 961.221323] nova-conductor[54431]: DEBUG nova.quota [None req-640f5a23-9c80-44b2-bb1c-3d22fef1d53e tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Getting quotas for user 7da3078e8bb5444c96e007cfc2d57952 and project c9f95d7e954f4413b01aad519c498967. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 961.451283] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 961.478777] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 961.478996] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 961.479208] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 961.521092] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 961.521319] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 961.521483] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 961.521840] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 961.522036] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 961.522187] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 961.529146] nova-conductor[54431]: DEBUG nova.quota [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Getting quotas for project c9f95d7e954f4413b01aad519c498967. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 961.531453] nova-conductor[54431]: DEBUG nova.quota [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Getting quotas for user 7da3078e8bb5444c96e007cfc2d57952 and project c9f95d7e954f4413b01aad519c498967. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 961.536838] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 7d91c659-d0ac-435c-af73-d6629bcde6bf] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 961.536838] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 961.536993] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 961.537184] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 961.542310] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 7d91c659-d0ac-435c-af73-d6629bcde6bf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 961.543065] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 961.543413] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 961.543413] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 961.556082] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 961.556293] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 961.556458] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 961.982756] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 99920efd-13e6-4a18-9532-4b53bea3c736] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 461e9d84-836a-4f8b-8ea2-f4ea9f208eef, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 99920efd-13e6-4a18-9532-4b53bea3c736 was re-scheduled: Binding failed for port 461e9d84-836a-4f8b-8ea2-f4ea9f208eef, please check neutron logs for more information.\n'] [ 961.983052] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 961.983240] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 99920efd-13e6-4a18-9532-4b53bea3c736.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 99920efd-13e6-4a18-9532-4b53bea3c736. [ 961.983556] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 99920efd-13e6-4a18-9532-4b53bea3c736] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 99920efd-13e6-4a18-9532-4b53bea3c736. [ 962.004863] nova-conductor[54431]: DEBUG nova.network.neutron [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 99920efd-13e6-4a18-9532-4b53bea3c736] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 962.024131] nova-conductor[54431]: DEBUG nova.network.neutron [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 99920efd-13e6-4a18-9532-4b53bea3c736] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 962.027087] nova-conductor[54431]: DEBUG nova.network.neutron [None req-ad9e2027-8ae3-4bdb-88a8-fe1e94427de8 tempest-AttachVolumeNegativeTest-31807712 tempest-AttachVolumeNegativeTest-31807712-project-member] [instance: 99920efd-13e6-4a18-9532-4b53bea3c736] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 963.683860] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] [instance: 4170a2f4-73ff-4bc6-9dc6-d2c4765c449d] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 5f7a9791-8f2a-445c-a05f-00d4c5bd13c5, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 4170a2f4-73ff-4bc6-9dc6-d2c4765c449d was re-scheduled: Binding failed for port 5f7a9791-8f2a-445c-a05f-00d4c5bd13c5, please check neutron logs for more information.\n'] [ 963.684560] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 963.684856] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4170a2f4-73ff-4bc6-9dc6-d2c4765c449d.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4170a2f4-73ff-4bc6-9dc6-d2c4765c449d. [ 963.685234] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] [instance: 4170a2f4-73ff-4bc6-9dc6-d2c4765c449d] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4170a2f4-73ff-4bc6-9dc6-d2c4765c449d. [ 963.727725] nova-conductor[54432]: DEBUG nova.network.neutron [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] [instance: 4170a2f4-73ff-4bc6-9dc6-d2c4765c449d] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 963.744517] nova-conductor[54432]: DEBUG nova.network.neutron [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] [instance: 4170a2f4-73ff-4bc6-9dc6-d2c4765c449d] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 963.748516] nova-conductor[54432]: DEBUG nova.network.neutron [None req-8aea0860-3bf9-4a17-bdf9-51230c24f2b8 tempest-ServersNegativeTestJSON-1007551573 tempest-ServersNegativeTestJSON-1007551573-project-member] [instance: 4170a2f4-73ff-4bc6-9dc6-d2c4765c449d] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 969.789338] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: ca937041-1aa6-46d8-be4e-b4983652fe15] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 438d8dc0-9244-438c-b078-fa7754182d12, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ca937041-1aa6-46d8-be4e-b4983652fe15 was re-scheduled: Binding failed for port 438d8dc0-9244-438c-b078-fa7754182d12, please check neutron logs for more information.\n'] [ 969.791634] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 969.791880] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ca937041-1aa6-46d8-be4e-b4983652fe15.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ca937041-1aa6-46d8-be4e-b4983652fe15. [ 969.792121] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: ca937041-1aa6-46d8-be4e-b4983652fe15] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ca937041-1aa6-46d8-be4e-b4983652fe15. [ 969.835068] nova-conductor[54432]: DEBUG nova.network.neutron [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: ca937041-1aa6-46d8-be4e-b4983652fe15] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 969.860486] nova-conductor[54432]: DEBUG nova.network.neutron [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: ca937041-1aa6-46d8-be4e-b4983652fe15] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 969.863908] nova-conductor[54432]: DEBUG nova.network.neutron [None req-1e3aa116-f163-4221-b591-3b122d27a0a6 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: ca937041-1aa6-46d8-be4e-b4983652fe15] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 970.374039] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell0:SELECT=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 970.395101] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=6,nova_cell1:INSERT=14 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 970.402788] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 970.404953] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=11,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 971.479193] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-79355559-cac2-429d-b908-23be3f534cad tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 971.489710] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-79355559-cac2-429d-b908-23be3f534cad tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: 92efc0d4-f11f-4ae6-9dbf-882d2cc0d4df] While scheduling instance, the build request was already deleted. {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1706}} [ 971.520718] nova-conductor[54432]: INFO nova.scheduler.client.report [None req-79355559-cac2-429d-b908-23be3f534cad tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Deleted allocations for instance 92efc0d4-f11f-4ae6-9dbf-882d2cc0d4df [ 971.525427] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-79355559-cac2-429d-b908-23be3f534cad tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 971.525777] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell0:SELECT=4 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 971.525997] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-79355559-cac2-429d-b908-23be3f534cad tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 971.526132] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-79355559-cac2-429d-b908-23be3f534cad tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 971.526496] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-79355559-cac2-429d-b908-23be3f534cad tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 971.526679] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-79355559-cac2-429d-b908-23be3f534cad tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 971.526834] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-79355559-cac2-429d-b908-23be3f534cad tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 971.534534] nova-conductor[54432]: DEBUG nova.quota [None req-79355559-cac2-429d-b908-23be3f534cad tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Getting quotas for project f509dfe537d441bcacb1a4aab3607214. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 971.537309] nova-conductor[54432]: DEBUG nova.quota [None req-79355559-cac2-429d-b908-23be3f534cad tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Getting quotas for user 02640299eb9c4a65bc80de6242dd2cd5 and project f509dfe537d441bcacb1a4aab3607214. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 971.545657] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=10,nova_cell1:SAVEPOINT=2,nova_cell1:INSERT=23,nova_cell1:RELEASE=2 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 971.554223] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=24,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 971.555821] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=21,nova_api:UPDATE=2,nova_api:DELETE=3 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 971.936225] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 971.950015] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 971.950267] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 971.950439] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 972.013759] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 972.013999] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 972.014178] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 972.014539] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 972.014711] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 972.014861] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 972.031178] nova-conductor[54432]: DEBUG nova.quota [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Getting quotas for project f509dfe537d441bcacb1a4aab3607214. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 972.035300] nova-conductor[54432]: DEBUG nova.quota [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Getting quotas for user 02640299eb9c4a65bc80de6242dd2cd5 and project f509dfe537d441bcacb1a4aab3607214. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 972.041107] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: fd7ac6e2-5033-4257-b8c4-cff1c56ca8ab] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 972.042297] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 972.042297] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 972.042297] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 972.049305] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: fd7ac6e2-5033-4257-b8c4-cff1c56ca8ab] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 972.050009] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 972.050234] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 972.050398] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 972.067364] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 972.067684] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 972.070321] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 980.676967] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] [instance: 636d27e5-f184-4656-9225-09c6ec0f0772] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 92271e85-7991-4870-b66b-67bbb7cfdd5c, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 636d27e5-f184-4656-9225-09c6ec0f0772 was re-scheduled: Binding failed for port 92271e85-7991-4870-b66b-67bbb7cfdd5c, please check neutron logs for more information.\n'] [ 980.677945] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 980.678308] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 636d27e5-f184-4656-9225-09c6ec0f0772.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 636d27e5-f184-4656-9225-09c6ec0f0772. [ 980.678804] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] [instance: 636d27e5-f184-4656-9225-09c6ec0f0772] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 636d27e5-f184-4656-9225-09c6ec0f0772. [ 980.717390] nova-conductor[54432]: DEBUG nova.network.neutron [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] [instance: 636d27e5-f184-4656-9225-09c6ec0f0772] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 980.733536] nova-conductor[54432]: DEBUG nova.network.neutron [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] [instance: 636d27e5-f184-4656-9225-09c6ec0f0772] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 980.739211] nova-conductor[54432]: DEBUG nova.network.neutron [None req-f2c542da-4b8d-471b-be4d-3a37acc71230 tempest-ServerMetadataTestJSON-949477722 tempest-ServerMetadataTestJSON-949477722-project-member] [instance: 636d27e5-f184-4656-9225-09c6ec0f0772] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 982.023357] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell0:SELECT=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 982.052563] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=4,nova_cell1:INSERT=7 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 982.060745] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 982.064600] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=8,nova_api:DELETE=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 982.349628] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: b4e185e6-b177-4212-bad8-b9fd1b51ea95] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b92e7d6b-5c1d-4570-ab7e-15c081e63812, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance b4e185e6-b177-4212-bad8-b9fd1b51ea95 was re-scheduled: Binding failed for port b92e7d6b-5c1d-4570-ab7e-15c081e63812, please check neutron logs for more information.\n'] [ 982.355125] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 982.355125] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b4e185e6-b177-4212-bad8-b9fd1b51ea95.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b4e185e6-b177-4212-bad8-b9fd1b51ea95. [ 982.355125] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: b4e185e6-b177-4212-bad8-b9fd1b51ea95] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b4e185e6-b177-4212-bad8-b9fd1b51ea95. [ 982.394834] nova-conductor[54431]: DEBUG nova.network.neutron [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: b4e185e6-b177-4212-bad8-b9fd1b51ea95] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 982.414067] nova-conductor[54431]: DEBUG nova.network.neutron [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: b4e185e6-b177-4212-bad8-b9fd1b51ea95] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 982.417758] nova-conductor[54431]: DEBUG nova.network.neutron [None req-67f41cfa-a853-430d-aa7a-71084176d495 tempest-ImagesTestJSON-1154998582 tempest-ImagesTestJSON-1154998582-project-member] [instance: b4e185e6-b177-4212-bad8-b9fd1b51ea95] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 986.277942] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] [instance: d9151031-e57a-4bc9-9f6f-ce2d8e74a001] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 84cefa2b-4e20-45df-a4c6-1b623a06142c, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance d9151031-e57a-4bc9-9f6f-ce2d8e74a001 was re-scheduled: Binding failed for port 84cefa2b-4e20-45df-a4c6-1b623a06142c, please check neutron logs for more information.\n'] [ 986.278590] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 986.278821] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d9151031-e57a-4bc9-9f6f-ce2d8e74a001.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d9151031-e57a-4bc9-9f6f-ce2d8e74a001. [ 986.279047] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] [instance: d9151031-e57a-4bc9-9f6f-ce2d8e74a001] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance d9151031-e57a-4bc9-9f6f-ce2d8e74a001. [ 986.300775] nova-conductor[54431]: DEBUG nova.network.neutron [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] [instance: d9151031-e57a-4bc9-9f6f-ce2d8e74a001] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 986.316128] nova-conductor[54431]: DEBUG nova.network.neutron [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] [instance: d9151031-e57a-4bc9-9f6f-ce2d8e74a001] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 986.318653] nova-conductor[54431]: DEBUG nova.network.neutron [None req-57946dbc-1615-4927-b014-841c3deeb332 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] [instance: d9151031-e57a-4bc9-9f6f-ce2d8e74a001] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 987.865065] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 1c1379c2-348b-4be8-9415-594ce53ebf55] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 9d32fadd-a63a-4b82-a49c-297505381d88, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 1c1379c2-348b-4be8-9415-594ce53ebf55 was re-scheduled: Binding failed for port 9d32fadd-a63a-4b82-a49c-297505381d88, please check neutron logs for more information.\n'] [ 987.865674] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 987.865896] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 1c1379c2-348b-4be8-9415-594ce53ebf55.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 1c1379c2-348b-4be8-9415-594ce53ebf55. [ 987.866118] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 1c1379c2-348b-4be8-9415-594ce53ebf55] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 1c1379c2-348b-4be8-9415-594ce53ebf55. [ 987.890906] nova-conductor[54432]: DEBUG nova.network.neutron [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 1c1379c2-348b-4be8-9415-594ce53ebf55] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 987.906142] nova-conductor[54432]: DEBUG nova.network.neutron [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 1c1379c2-348b-4be8-9415-594ce53ebf55] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 987.909353] nova-conductor[54432]: DEBUG nova.network.neutron [None req-2348a2d7-652a-404b-9527-dbdb81c0306d tempest-ServerDiskConfigTestJSON-1033696558 tempest-ServerDiskConfigTestJSON-1033696558-project-member] [instance: 1c1379c2-348b-4be8-9415-594ce53ebf55] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 988.802585] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 988.815781] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 988.816024] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 988.816204] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 988.845127] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 988.845414] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 988.845586] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 988.845961] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 988.846158] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 988.846317] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 988.853488] nova-conductor[54431]: DEBUG nova.quota [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Getting quotas for project c21dbc475b0d4225b920fbad495b785a. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 988.855582] nova-conductor[54431]: DEBUG nova.quota [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Getting quotas for user 48b2a1fbea0e4e22bfa32ec31e438f6d and project c21dbc475b0d4225b920fbad495b785a. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 988.860371] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] [instance: 0cd2c169-ee86-4664-b134-a6c35bde6764] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 988.860801] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 988.860998] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 988.861196] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 988.865838] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] [instance: 0cd2c169-ee86-4664-b134-a6c35bde6764] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 988.866475] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 988.866673] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 988.866835] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 988.882077] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 988.882283] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 988.882449] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 989.517402] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 7d91c659-d0ac-435c-af73-d6629bcde6bf] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 613001b4-83a5-4226-a98a-9814c2d78697, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 7d91c659-d0ac-435c-af73-d6629bcde6bf was re-scheduled: Binding failed for port 613001b4-83a5-4226-a98a-9814c2d78697, please check neutron logs for more information.\n'] [ 989.517914] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 989.518225] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7d91c659-d0ac-435c-af73-d6629bcde6bf.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7d91c659-d0ac-435c-af73-d6629bcde6bf. [ 989.518560] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 7d91c659-d0ac-435c-af73-d6629bcde6bf] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7d91c659-d0ac-435c-af73-d6629bcde6bf. [ 989.536795] nova-conductor[54431]: DEBUG nova.network.neutron [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 7d91c659-d0ac-435c-af73-d6629bcde6bf] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 989.552757] nova-conductor[54431]: DEBUG nova.network.neutron [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 7d91c659-d0ac-435c-af73-d6629bcde6bf] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 989.555655] nova-conductor[54431]: DEBUG nova.network.neutron [None req-5d303a48-fb9a-4ec6-adf5-7102764bb3d0 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 7d91c659-d0ac-435c-af73-d6629bcde6bf] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 990.649967] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 990.662454] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 990.662638] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 990.662800] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 990.690737] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 990.690961] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 990.691178] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 990.691547] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 990.691767] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 990.692044] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 990.701605] nova-conductor[54432]: DEBUG nova.quota [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Getting quotas for project c9f95d7e954f4413b01aad519c498967. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 990.704222] nova-conductor[54432]: DEBUG nova.quota [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Getting quotas for user 7da3078e8bb5444c96e007cfc2d57952 and project c9f95d7e954f4413b01aad519c498967. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 990.708994] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 9d9196c8-3e26-4223-ab13-af4bcd31787b] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 990.709417] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 990.709613] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 990.709773] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 990.715971] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 9d9196c8-3e26-4223-ab13-af4bcd31787b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 990.716634] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 990.716834] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 990.716995] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 990.739711] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 990.739913] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 990.740097] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 991.130214] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: fd7ac6e2-5033-4257-b8c4-cff1c56ca8ab] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port e1b1c46e-1478-45a2-9e15-0b8ddcd44cb4, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance fd7ac6e2-5033-4257-b8c4-cff1c56ca8ab was re-scheduled: Binding failed for port e1b1c46e-1478-45a2-9e15-0b8ddcd44cb4, please check neutron logs for more information.\n'] [ 991.130848] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 991.131108] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fd7ac6e2-5033-4257-b8c4-cff1c56ca8ab.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fd7ac6e2-5033-4257-b8c4-cff1c56ca8ab. [ 991.131344] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: fd7ac6e2-5033-4257-b8c4-cff1c56ca8ab] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fd7ac6e2-5033-4257-b8c4-cff1c56ca8ab. [ 991.156623] nova-conductor[54431]: DEBUG nova.network.neutron [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: fd7ac6e2-5033-4257-b8c4-cff1c56ca8ab] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 991.171934] nova-conductor[54431]: DEBUG nova.network.neutron [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: fd7ac6e2-5033-4257-b8c4-cff1c56ca8ab] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 991.175120] nova-conductor[54431]: DEBUG nova.network.neutron [None req-b9a61fbf-a336-4e31-afc6-1f3919c80a99 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: fd7ac6e2-5033-4257-b8c4-cff1c56ca8ab] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 992.807900] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Took 0.51 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 992.820411] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 992.820617] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 992.820786] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 992.843966] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 992.844190] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 992.844357] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 992.844711] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 992.844894] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 992.845064] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 992.852109] nova-conductor[54432]: DEBUG nova.quota [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Getting quotas for project f509dfe537d441bcacb1a4aab3607214. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 992.854196] nova-conductor[54432]: DEBUG nova.quota [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Getting quotas for user 02640299eb9c4a65bc80de6242dd2cd5 and project f509dfe537d441bcacb1a4aab3607214. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 992.858853] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: 3e1fbc16-5a38-4fe4-aa5b-b7982b81e7a2] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 992.859292] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 992.859486] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 992.859644] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 992.863881] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: 3e1fbc16-5a38-4fe4-aa5b-b7982b81e7a2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 992.864473] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 992.864664] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 992.864827] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 992.876529] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 992.876721] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 992.876882] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 996.803429] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=398,nova_cell1:UPDATE=211,nova_cell1:SAVEPOINT=40,nova_cell1:RELEASE=40,nova_cell1:DELETE=2,nova_cell1:INSERT=31 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 998.849978] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell0:SELECT=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 998.868952] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=3,nova_cell1:INSERT=7 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 998.877220] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=6 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 998.879135] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=6,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1002.168055] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=339,nova_cell1:UPDATE=157,nova_cell1:SAVEPOINT=36,nova_cell1:RELEASE=36,nova_cell1:INSERT=20 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1002.848449] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell0:SELECT=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1002.867225] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=6,nova_cell1:INSERT=14 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1002.872655] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=12 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1002.875919] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=12,nova_api:UPDATE=2,nova_api:DELETE=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1004.721521] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] [instance: 0cd2c169-ee86-4664-b134-a6c35bde6764] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 16bf4b36-3e68-4e34-9943-51e8c0fe805a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 0cd2c169-ee86-4664-b134-a6c35bde6764 was re-scheduled: Binding failed for port 16bf4b36-3e68-4e34-9943-51e8c0fe805a, please check neutron logs for more information.\n'] [ 1004.722073] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1004.722304] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0cd2c169-ee86-4664-b134-a6c35bde6764.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0cd2c169-ee86-4664-b134-a6c35bde6764. [ 1004.722516] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] [instance: 0cd2c169-ee86-4664-b134-a6c35bde6764] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0cd2c169-ee86-4664-b134-a6c35bde6764. [ 1004.744247] nova-conductor[54431]: DEBUG nova.network.neutron [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] [instance: 0cd2c169-ee86-4664-b134-a6c35bde6764] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 1004.759179] nova-conductor[54431]: DEBUG nova.network.neutron [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] [instance: 0cd2c169-ee86-4664-b134-a6c35bde6764] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 1004.761940] nova-conductor[54431]: DEBUG nova.network.neutron [None req-66aef1cf-2289-4e5d-ae5b-3280a8f721c8 tempest-AttachVolumeShelveTestJSON-2006650516 tempest-AttachVolumeShelveTestJSON-2006650516-project-member] [instance: 0cd2c169-ee86-4664-b134-a6c35bde6764] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1006.427444] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 9d9196c8-3e26-4223-ab13-af4bcd31787b] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 846f3831-1fe9-4a19-96b3-87096dd5a2d4, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 9d9196c8-3e26-4223-ab13-af4bcd31787b was re-scheduled: Binding failed for port 846f3831-1fe9-4a19-96b3-87096dd5a2d4, please check neutron logs for more information.\n'] [ 1006.428065] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1006.428307] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9d9196c8-3e26-4223-ab13-af4bcd31787b.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9d9196c8-3e26-4223-ab13-af4bcd31787b. [ 1006.428539] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 9d9196c8-3e26-4223-ab13-af4bcd31787b] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9d9196c8-3e26-4223-ab13-af4bcd31787b. [ 1006.453629] nova-conductor[54431]: DEBUG nova.network.neutron [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 9d9196c8-3e26-4223-ab13-af4bcd31787b] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 1006.472803] nova-conductor[54431]: DEBUG nova.network.neutron [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 9d9196c8-3e26-4223-ab13-af4bcd31787b] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 1006.476128] nova-conductor[54431]: DEBUG nova.network.neutron [None req-b2c92193-b4f4-4fe3-9aa5-d1bb048cb0f5 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 9d9196c8-3e26-4223-ab13-af4bcd31787b] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1008.168331] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1008.183827] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1008.184055] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1008.184229] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1008.208306] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1008.208486] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1008.208647] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1008.209017] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1008.209219] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1008.209387] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1008.217770] nova-conductor[54432]: DEBUG nova.quota [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Getting quotas for project c9f95d7e954f4413b01aad519c498967. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1008.220463] nova-conductor[54432]: DEBUG nova.quota [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Getting quotas for user 7da3078e8bb5444c96e007cfc2d57952 and project c9f95d7e954f4413b01aad519c498967. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1008.225344] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 607a5220-597e-4480-ae4d-150802c6196a] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1008.225780] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1008.225970] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1008.226149] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1008.229718] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 607a5220-597e-4480-ae4d-150802c6196a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1008.230343] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1008.230535] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1008.230701] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1008.248552] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1008.248810] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1008.249023] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1008.692143] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: 3e1fbc16-5a38-4fe4-aa5b-b7982b81e7a2] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 334799b5-89ef-45c5-b835-f2b50f55d337, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 3e1fbc16-5a38-4fe4-aa5b-b7982b81e7a2 was re-scheduled: Binding failed for port 334799b5-89ef-45c5-b835-f2b50f55d337, please check neutron logs for more information.\n'] [ 1008.692801] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1008.693041] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3e1fbc16-5a38-4fe4-aa5b-b7982b81e7a2.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3e1fbc16-5a38-4fe4-aa5b-b7982b81e7a2. [ 1008.693268] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: 3e1fbc16-5a38-4fe4-aa5b-b7982b81e7a2] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3e1fbc16-5a38-4fe4-aa5b-b7982b81e7a2. [ 1008.713451] nova-conductor[54431]: DEBUG nova.network.neutron [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: 3e1fbc16-5a38-4fe4-aa5b-b7982b81e7a2] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 1008.729562] nova-conductor[54431]: DEBUG nova.network.neutron [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: 3e1fbc16-5a38-4fe4-aa5b-b7982b81e7a2] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 1008.732917] nova-conductor[54431]: DEBUG nova.network.neutron [None req-be2c283d-b325-4dca-869f-2c08b53210c3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: 3e1fbc16-5a38-4fe4-aa5b-b7982b81e7a2] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1010.347605] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1010.359612] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1010.359826] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1010.359989] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1010.383535] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1010.383750] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1010.383916] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1010.384398] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1010.384524] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1010.384682] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1010.394194] nova-conductor[54432]: DEBUG nova.quota [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Getting quotas for project f509dfe537d441bcacb1a4aab3607214. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1010.396833] nova-conductor[54432]: DEBUG nova.quota [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Getting quotas for user 02640299eb9c4a65bc80de6242dd2cd5 and project f509dfe537d441bcacb1a4aab3607214. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1010.401609] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: b4b02e6d-276d-4271-8d4f-3891cfaec8af] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1010.402084] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1010.402288] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1010.402471] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1010.405806] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: b4b02e6d-276d-4271-8d4f-3891cfaec8af] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1010.406402] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1010.406595] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1010.406756] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1010.420609] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1010.420797] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1010.420956] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1020.389663] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell0:SELECT=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1020.408811] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=6,nova_cell1:INSERT=14 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1020.416674] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=12,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1020.418468] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=12,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1020.796629] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 607a5220-597e-4480-ae4d-150802c6196a] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 4ed1776b-c57d-4da6-a8f3-b15b2561f300, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 607a5220-597e-4480-ae4d-150802c6196a was re-scheduled: Binding failed for port 4ed1776b-c57d-4da6-a8f3-b15b2561f300, please check neutron logs for more information.\n'] [ 1020.797258] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1020.797412] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 607a5220-597e-4480-ae4d-150802c6196a.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 607a5220-597e-4480-ae4d-150802c6196a. [ 1020.797727] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 607a5220-597e-4480-ae4d-150802c6196a] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 607a5220-597e-4480-ae4d-150802c6196a. [ 1020.823854] nova-conductor[54432]: DEBUG nova.network.neutron [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 607a5220-597e-4480-ae4d-150802c6196a] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 1020.840267] nova-conductor[54432]: DEBUG nova.network.neutron [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 607a5220-597e-4480-ae4d-150802c6196a] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 1020.843779] nova-conductor[54432]: DEBUG nova.network.neutron [None req-46bb1efe-5c23-4d52-bd29-24c167cdb74d tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 607a5220-597e-4480-ae4d-150802c6196a] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1022.433057] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1022.445320] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1022.445537] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1022.445702] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1022.473950] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1022.474180] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1022.474346] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1022.474698] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1022.474875] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1022.475039] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1022.483838] nova-conductor[54432]: DEBUG nova.quota [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Getting quotas for project c9f95d7e954f4413b01aad519c498967. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1022.486457] nova-conductor[54432]: DEBUG nova.quota [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Getting quotas for user 7da3078e8bb5444c96e007cfc2d57952 and project c9f95d7e954f4413b01aad519c498967. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1022.491210] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 86fcebad-2773-4e53-b745-2fd6acaf4756] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1022.491609] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1022.491798] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1022.491956] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1022.495028] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 86fcebad-2773-4e53-b745-2fd6acaf4756] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1022.495622] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1022.495813] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1022.495972] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1022.510501] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1022.510701] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1022.510863] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1022.533709] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: b4b02e6d-276d-4271-8d4f-3891cfaec8af] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port ac4cba57-ba76-4654-ad50-36ee9a68f1ef, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance b4b02e6d-276d-4271-8d4f-3891cfaec8af was re-scheduled: Binding failed for port ac4cba57-ba76-4654-ad50-36ee9a68f1ef, please check neutron logs for more information.\n'] [ 1022.533709] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1022.533709] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b4b02e6d-276d-4271-8d4f-3891cfaec8af.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b4b02e6d-276d-4271-8d4f-3891cfaec8af. [ 1022.534105] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: b4b02e6d-276d-4271-8d4f-3891cfaec8af] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b4b02e6d-276d-4271-8d4f-3891cfaec8af. [ 1022.552906] nova-conductor[54431]: DEBUG nova.network.neutron [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: b4b02e6d-276d-4271-8d4f-3891cfaec8af] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 1022.575705] nova-conductor[54431]: DEBUG nova.network.neutron [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: b4b02e6d-276d-4271-8d4f-3891cfaec8af] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 1022.578908] nova-conductor[54431]: DEBUG nova.network.neutron [None req-41371a42-7802-4cdc-af6b-98cb6e9ef16b tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: b4b02e6d-276d-4271-8d4f-3891cfaec8af] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1024.650079] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54432) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1024.663292] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1024.663518] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1024.663685] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1024.685710] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1024.685952] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1024.686156] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1024.686562] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1024.686747] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1024.686905] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1024.694145] nova-conductor[54432]: DEBUG nova.quota [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Getting quotas for project f509dfe537d441bcacb1a4aab3607214. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1024.696267] nova-conductor[54432]: DEBUG nova.quota [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Getting quotas for user 02640299eb9c4a65bc80de6242dd2cd5 and project f509dfe537d441bcacb1a4aab3607214. Resources: {'instances', 'cores', 'ram'} {{(pid=54432) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1024.701170] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: e2733e84-0140-4b7f-8a7b-bdbb079d09b5] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54432) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1024.701604] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1024.701795] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1024.701955] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1024.705626] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: e2733e84-0140-4b7f-8a7b-bdbb079d09b5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54432) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1024.706226] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1024.706419] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1024.706582] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1024.719605] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1024.719800] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1024.719963] nova-conductor[54432]: DEBUG oslo_concurrency.lockutils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54432) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1034.690773] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell0:SELECT=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1034.708457] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SELECT=6,nova_cell1:INSERT=14 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1034.714331] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1034.717781] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_api:SELECT=11,nova_api:DELETE=2 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1035.080121] nova-conductor[54432]: ERROR nova.scheduler.utils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 86fcebad-2773-4e53-b745-2fd6acaf4756] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 69d32c23-6404-40bf-9165-725175531e05, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 86fcebad-2773-4e53-b745-2fd6acaf4756 was re-scheduled: Binding failed for port 69d32c23-6404-40bf-9165-725175531e05, please check neutron logs for more information.\n'] [ 1035.080836] nova-conductor[54432]: DEBUG nova.conductor.manager [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Rescheduling: True {{(pid=54432) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1035.081659] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 86fcebad-2773-4e53-b745-2fd6acaf4756.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 86fcebad-2773-4e53-b745-2fd6acaf4756. [ 1035.082056] nova-conductor[54432]: WARNING nova.scheduler.utils [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 86fcebad-2773-4e53-b745-2fd6acaf4756] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 86fcebad-2773-4e53-b745-2fd6acaf4756. [ 1035.105057] nova-conductor[54432]: DEBUG nova.network.neutron [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 86fcebad-2773-4e53-b745-2fd6acaf4756] deallocate_for_instance() {{(pid=54432) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 1035.120282] nova-conductor[54432]: DEBUG nova.network.neutron [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 86fcebad-2773-4e53-b745-2fd6acaf4756] Instance cache missing network info. {{(pid=54432) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 1035.122992] nova-conductor[54432]: DEBUG nova.network.neutron [None req-483b3412-9b53-4902-8762-0f8b053c02b9 tempest-DeleteServersTestJSON-1536511657 tempest-DeleteServersTestJSON-1536511657-project-member] [instance: 86fcebad-2773-4e53-b745-2fd6acaf4756] Updating instance_info_cache with network_info: [] {{(pid=54432) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1036.869295] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: e2733e84-0140-4b7f-8a7b-bdbb079d09b5] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 28608160-16ec-4af3-93f3-843fe7d0630b, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance e2733e84-0140-4b7f-8a7b-bdbb079d09b5 was re-scheduled: Binding failed for port 28608160-16ec-4af3-93f3-843fe7d0630b, please check neutron logs for more information.\n'] [ 1036.869915] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1036.870169] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e2733e84-0140-4b7f-8a7b-bdbb079d09b5.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e2733e84-0140-4b7f-8a7b-bdbb079d09b5. [ 1036.870435] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: e2733e84-0140-4b7f-8a7b-bdbb079d09b5] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e2733e84-0140-4b7f-8a7b-bdbb079d09b5. [ 1036.894619] nova-conductor[54431]: DEBUG nova.network.neutron [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: e2733e84-0140-4b7f-8a7b-bdbb079d09b5] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 1036.913984] nova-conductor[54431]: DEBUG nova.network.neutron [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: e2733e84-0140-4b7f-8a7b-bdbb079d09b5] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 1036.917637] nova-conductor[54431]: DEBUG nova.network.neutron [None req-67a0b9ec-bc6d-4e10-a461-fd1106abccd3 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: e2733e84-0140-4b7f-8a7b-bdbb079d09b5] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1038.923922] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Took 0.50 seconds to select destinations for 1 instance(s). {{(pid=54431) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 1038.937851] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1038.938112] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1038.938320] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1038.966467] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1038.966717] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1038.966887] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1038.967272] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1038.967456] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1038.967664] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1038.975230] nova-conductor[54431]: DEBUG nova.quota [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Getting quotas for project f509dfe537d441bcacb1a4aab3607214. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 1038.977432] nova-conductor[54431]: DEBUG nova.quota [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Getting quotas for user 02640299eb9c4a65bc80de6242dd2cd5 and project f509dfe537d441bcacb1a4aab3607214. Resources: {'instances', 'cores', 'ram'} {{(pid=54431) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 1038.982292] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: 6cdc499b-8441-48c6-a6e7-07383e5746db] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=54431) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 1038.982674] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1038.982868] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1038.983037] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1038.987168] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: 6cdc499b-8441-48c6-a6e7-07383e5746db] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0422123c-16ce-4c9d-b1c6-0281fe0e54ad',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=54431) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 1038.987708] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1038.987903] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1038.988080] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1039.001051] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Acquiring lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:402}} [ 1039.001275] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:407}} [ 1039.001493] nova-conductor[54431]: DEBUG oslo_concurrency.lockutils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Lock "8fb5f300-ab58-4d7c-afe8-8d0b4be88dee" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=54431) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}} [ 1048.971651] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell0:SELECT=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1048.990058] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=3,nova_cell1:INSERT=7 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1048.997128] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=7,nova_api:UPDATE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1048.999694] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_api:SELECT=5,nova_api:DELETE=1 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1051.509980] nova-conductor[54431]: ERROR nova.scheduler.utils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: 6cdc499b-8441-48c6-a6e7-07383e5746db] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2641, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 546, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 225, in wait\n return self._exit_event.wait()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/event.py", line 131, in wait\n current.throw(*self._exc)\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 663, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2011, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1989, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1278, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1414, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1389, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 15adb248-7814-4c18-b4d0-e725f0cb79e3, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2456, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2748, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 6cdc499b-8441-48c6-a6e7-07383e5746db was re-scheduled: Binding failed for port 15adb248-7814-4c18-b4d0-e725f0cb79e3, please check neutron logs for more information.\n'] [ 1051.510636] nova-conductor[54431]: DEBUG nova.conductor.manager [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Rescheduling: True {{(pid=54431) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 1051.510636] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6cdc499b-8441-48c6-a6e7-07383e5746db.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6cdc499b-8441-48c6-a6e7-07383e5746db. [ 1051.510636] nova-conductor[54431]: WARNING nova.scheduler.utils [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: 6cdc499b-8441-48c6-a6e7-07383e5746db] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6cdc499b-8441-48c6-a6e7-07383e5746db. [ 1051.530283] nova-conductor[54431]: DEBUG nova.network.neutron [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: 6cdc499b-8441-48c6-a6e7-07383e5746db] deallocate_for_instance() {{(pid=54431) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1860}} [ 1051.545449] nova-conductor[54431]: DEBUG nova.network.neutron [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: 6cdc499b-8441-48c6-a6e7-07383e5746db] Instance cache missing network info. {{(pid=54431) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3380}} [ 1051.548466] nova-conductor[54431]: DEBUG nova.network.neutron [None req-619d1589-e774-47bb-ae60-3ac15c3f2d75 tempest-ServersTestJSON-497234563 tempest-ServersTestJSON-497234563-project-member] [instance: 6cdc499b-8441-48c6-a6e7-07383e5746db] Updating instance_info_cache with network_info: [] {{(pid=54431) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 1065.316908] nova-conductor[54431]: DEBUG dbcounter [-] [54431] Writing DB stats nova_cell1:SELECT=199,nova_cell1:UPDATE=102,nova_cell1:SAVEPOINT=24,nova_cell1:RELEASE=24,nova_cell1:INSERT=13 {{(pid=54431) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}} [ 1065.824897] nova-conductor[54432]: DEBUG dbcounter [-] [54432] Writing DB stats nova_cell1:SAVEPOINT=23,nova_cell1:RELEASE=23,nova_cell1:SELECT=173,nova_cell1:UPDATE=79,nova_cell1:INSERT=14 {{(pid=54432) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:115}}