[ 489.186362] nova-conductor[51822]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. [ 490.408954] nova-conductor[51822]: DEBUG oslo_db.sqlalchemy.engines [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=51822) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 490.434562] nova-conductor[51822]: DEBUG nova.context [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),74f912f4-43d8-4d2a-9ea1-d6a83c370e35(cell1) {{(pid=51822) load_cells /opt/stack/nova/nova/context.py:464}} [ 490.436309] nova-conductor[51822]: DEBUG oslo_concurrency.lockutils [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=51822) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 490.436516] nova-conductor[51822]: DEBUG oslo_concurrency.lockutils [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=51822) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 490.436955] nova-conductor[51822]: DEBUG oslo_concurrency.lockutils [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=51822) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 490.437296] nova-conductor[51822]: DEBUG oslo_concurrency.lockutils [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=51822) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 490.437481] nova-conductor[51822]: DEBUG oslo_concurrency.lockutils [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=51822) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 490.438370] nova-conductor[51822]: DEBUG oslo_concurrency.lockutils [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=51822) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 490.443802] nova-conductor[51822]: DEBUG oslo_db.sqlalchemy.engines [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=51822) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 490.444177] nova-conductor[51822]: DEBUG oslo_db.sqlalchemy.engines [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=51822) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 490.503288] nova-conductor[51822]: DEBUG oslo_concurrency.lockutils [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] Acquiring lock "singleton_lock" {{(pid=51822) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 490.503474] nova-conductor[51822]: DEBUG oslo_concurrency.lockutils [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] Acquired lock "singleton_lock" {{(pid=51822) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 490.503673] nova-conductor[51822]: DEBUG oslo_concurrency.lockutils [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] Releasing lock "singleton_lock" {{(pid=51822) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 490.504092] nova-conductor[51822]: INFO oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] Starting 2 workers [ 490.508469] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] Started child 52243 {{(pid=51822) _start_child /usr/local/lib/python3.10/dist-packages/oslo_service/service.py:575}} [ 490.511826] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] Started child 52244 {{(pid=51822) _start_child /usr/local/lib/python3.10/dist-packages/oslo_service/service.py:575}} [ 490.512511] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] Full set of CONF: {{(pid=51822) wait /usr/local/lib/python3.10/dist-packages/oslo_service/service.py:649}} [ 490.512723] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ******************************************************************************** {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2589}} [ 490.512879] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] Configuration options gathered from: {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2590}} [ 490.513064] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] command line args: ['--config-file', '/etc/nova/nova.conf'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2591}} [ 490.513412] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] config files: ['/etc/nova/nova.conf'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2592}} [ 490.513552] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ================================================================================ {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2594}} [ 490.513956] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] allow_resize_to_same_host = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.514185] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] arq_binding_timeout = 300 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.514238] nova-conductor[52243]: INFO nova.service [-] Starting conductor node (version 0.1.0) [ 490.514468] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] block_device_allocate_retries = 60 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.514560] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] block_device_allocate_retries_interval = 3 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.514781] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cert = self.pem {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.514974] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] compute_driver = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.515101] nova-conductor[52244]: INFO nova.service [-] Starting conductor node (version 0.1.0) [ 490.515291] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] compute_monitors = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.515464] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] config_dir = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.515666] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] config_drive_format = iso9660 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.515806] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] config_file = ['/etc/nova/nova.conf'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.515999] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] config_source = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.516203] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] console_host = devstack {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.516391] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] control_exchange = nova {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.516589] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cpu_allocation_ratio = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.516760] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] daemon = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.516955] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] debug = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.517154] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] default_access_ip_network_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.517337] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] default_availability_zone = nova {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.517508] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] default_ephemeral_format = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.517786] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.517974] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] default_schedule_zone = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.518161] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] disk_allocation_ratio = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.518317] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] enable_new_services = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.518552] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] enabled_apis = ['osapi_compute'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.518741] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] enabled_ssl_apis = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.518931] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] flat_injected = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.519108] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] force_config_drive = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.519284] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] force_raw_images = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.519484] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] graceful_shutdown_timeout = 5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.519658] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] heal_instance_info_cache_interval = 60 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.520124] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] host = devstack {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.520343] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] initial_cpu_allocation_ratio = 4.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.520528] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] initial_disk_allocation_ratio = 1.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.520707] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] initial_ram_allocation_ratio = 1.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.520979] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.521173] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] instance_build_timeout = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.521353] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] instance_delete_interval = 300 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.521550] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] instance_format = [instance: %(uuid)s] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.521716] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] instance_name_template = instance-%08x {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.521886] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] instance_usage_audit = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.522071] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] instance_usage_audit_period = month {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.522239] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] instance_uuid_format = [instance: %(uuid)s] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.522422] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] instances_path = /opt/stack/data/nova/instances {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.522610] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] internal_service_availability_zone = internal {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.522780] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] key = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.522937] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] live_migration_retry_count = 30 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.523130] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] log_config_append = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.523294] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.523471] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] log_dir = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.523631] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] log_file = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.523758] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] log_options = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.523940] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] log_rotate_interval = 1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.524531] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] log_rotate_interval_type = days {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.524735] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] log_rotation_type = none {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.524870] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.524997] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.525179] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.525359] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.525485] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.525676] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] long_rpc_timeout = 1800 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.525832] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] max_concurrent_builds = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.525986] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] max_concurrent_live_migrations = 1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.526150] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] max_concurrent_snapshots = 5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.526320] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] max_local_block_devices = 3 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.526496] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] max_logfile_count = 30 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.526627] nova-conductor[52244]: DEBUG oslo_db.sqlalchemy.engines [None req-29ad9b13-cae7-4199-b8ea-a138d0427279 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=52244) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 490.526682] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] max_logfile_size_mb = 200 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.526829] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] maximum_instance_delete_attempts = 5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.526992] nova-conductor[52243]: DEBUG oslo_db.sqlalchemy.engines [None req-fc09e859-22e5-43a9-8e10-6f724bf20701 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=52243) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 490.527040] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] metadata_listen = 0.0.0.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.527235] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] metadata_listen_port = 8775 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.527417] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] metadata_workers = 2 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.527574] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] migrate_max_retries = -1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.527754] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] mkisofs_cmd = genisoimage {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.527960] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] my_block_storage_ip = 10.180.1.21 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.528103] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] my_ip = 10.180.1.21 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.528259] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] network_allocate_retries = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.528433] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.528599] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] osapi_compute_listen = 0.0.0.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.528764] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] osapi_compute_listen_port = 8774 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.528926] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] osapi_compute_unique_server_name_scope = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.529100] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] osapi_compute_workers = 2 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.529258] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] password_length = 12 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.529420] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] periodic_enable = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.529620] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] periodic_fuzzy_delay = 60 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.529823] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] pointer_model = usbtablet {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.530037] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] preallocate_images = none {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.530195] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] publish_errors = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.530328] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] pybasedir = /opt/stack/nova {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.530501] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ram_allocation_ratio = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.530675] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] rate_limit_burst = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.530839] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] rate_limit_except_level = CRITICAL {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.531030] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] rate_limit_interval = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.531199] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] reboot_timeout = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.531369] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] reclaim_instance_interval = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.531549] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] record = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.531708] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] reimage_timeout_per_gb = 20 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.531871] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] report_interval = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.532038] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] rescue_timeout = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.532198] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] reserved_host_cpus = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.532352] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] reserved_host_disk_mb = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.532506] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] reserved_host_memory_mb = 512 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.532720] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] reserved_huge_pages = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.532891] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] resize_confirm_window = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.533092] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] resize_fs_using_block_device = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.533292] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] resume_guests_state_on_host_boot = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.533499] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.534067] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] rpc_response_timeout = 60 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.534249] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] run_external_periodic_tasks = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.534419] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] running_deleted_instance_action = reap {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.534580] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] running_deleted_instance_poll_interval = 1800 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.534739] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] running_deleted_instance_timeout = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.534893] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] scheduler_instance_sync_interval = 120 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.535088] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] service_down_time = 60 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.535280] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] servicegroup_driver = db {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.535435] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] shelved_offload_time = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.535591] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] shelved_poll_interval = 3600 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.535759] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] shutdown_timeout = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.535915] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] source_is_ipv6 = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.536080] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ssl_only = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.536243] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] state_path = /opt/stack/data/nova {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.536399] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] sync_power_state_interval = 600 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.536552] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] sync_power_state_pool_size = 1000 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.536718] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] syslog_log_facility = LOG_USER {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.536872] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] tempdir = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.537032] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] timeout_nbd = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.537220] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] transport_url = **** {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.537379] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] update_resources_interval = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.537531] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] use_cow_images = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.537707] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] use_eventlog = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.537881] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] use_journal = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.538047] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] use_json = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.538202] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] use_rootwrap_daemon = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.538360] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] use_stderr = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.538516] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] use_syslog = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.538667] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vcpu_pin_set = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.538830] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vif_plugging_is_fatal = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.538993] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vif_plugging_timeout = 300 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.539205] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] virt_mkfs = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.539380] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] volume_usage_poll_interval = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.539585] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] watch_log_file = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.539783] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] web = /usr/share/spice-html5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 490.540065] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_concurrency.disable_process_locking = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.540278] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.540488] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.540652] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.540820] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_metrics.metrics_process_name = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.540985] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.541157] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.541384] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.auth_strategy = keystone {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.541605] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.compute_link_prefix = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.541804] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.541980] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.dhcp_domain = novalocal {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.542163] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.enable_instance_password = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.542325] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.glance_link_prefix = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.542515] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.instance_list_cells_batch_fixed_size = 100 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.542697] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.instance_list_cells_batch_strategy = distributed {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.542857] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.instance_list_per_project_cells = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.543028] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.list_records_by_skipping_down_cells = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.543439] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.local_metadata_per_cell = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.543439] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.max_limit = 1000 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.543515] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.metadata_cache_expiration = 15 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.543665] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.neutron_default_tenant_id = default {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.543827] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.use_forwarded_for = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.544024] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.use_neutron_default_nets = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.544200] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.vendordata_dynamic_connect_timeout = 5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.544362] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.vendordata_dynamic_failure_fatal = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.544546] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.vendordata_dynamic_read_timeout = 5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.544715] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.vendordata_dynamic_ssl_certfile = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.544879] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.vendordata_dynamic_targets = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.545072] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.vendordata_jsonfile_path = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.545270] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api.vendordata_providers = ['StaticJSON'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.545537] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.backend = dogpile.cache.memcached {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.545719] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.backend_argument = **** {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.545904] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.config_prefix = cache.oslo {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.546109] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.dead_timeout = 60.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.546269] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.debug_cache_backend = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.546426] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.enable_retry_client = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.546583] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.enable_socket_keepalive = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.546753] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.enabled = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.546940] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.expiration_time = 600 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.547108] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.hashclient_retry_attempts = 2 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.547308] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.hashclient_retry_delay = 1.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.547471] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.memcache_dead_retry = 300 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.547633] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.memcache_password = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.547795] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.547953] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.548124] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.memcache_pool_maxsize = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.548303] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.548464] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.memcache_sasl_enabled = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.548598] nova-conductor[52244]: DEBUG nova.service [None req-29ad9b13-cae7-4199-b8ea-a138d0427279 None None] Creating RPC server for service conductor {{(pid=52244) start /opt/stack/nova/nova/service.py:182}} [ 490.548634] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.memcache_servers = ['localhost:11211'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.548796] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.memcache_socket_timeout = 1.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.548957] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.memcache_username = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.549129] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.proxies = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.549287] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.retry_attempts = 2 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.549449] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.retry_delay = 0.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.549626] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.socket_keepalive_count = 1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.549795] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.socket_keepalive_idle = 1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.549953] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.socket_keepalive_interval = 1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.550124] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.tls_allowed_ciphers = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.550281] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.tls_cafile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.550431] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.tls_certfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.550588] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.tls_enabled = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.550738] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cache.tls_keyfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.550954] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cinder.auth_section = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.551154] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cinder.auth_type = password {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.551321] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cinder.cafile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.551507] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cinder.catalog_info = volumev3::publicURL {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.551712] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cinder.certfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.551907] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cinder.collect_timing = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.552105] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cinder.cross_az_attach = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.552287] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cinder.debug = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.552445] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cinder.endpoint_template = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.552724] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cinder.http_retries = 3 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.552891] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cinder.insecure = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.553059] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cinder.keyfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.553227] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cinder.os_region_name = RegionOne {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.553386] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cinder.split_loggers = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.553539] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cinder.timeout = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.553730] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] compute.consecutive_build_service_disable_threshold = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.553892] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] compute.cpu_dedicated_set = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.554060] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] compute.cpu_shared_set = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.554223] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] compute.image_type_exclude_list = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.554387] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] compute.live_migration_wait_for_vif_plug = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.554548] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] compute.max_concurrent_disk_ops = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.554709] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] compute.max_disk_devices_to_attach = -1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.554867] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.555060] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.555222] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] compute.resource_provider_association_refresh = 300 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.555382] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] compute.shutdown_retry_interval = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.555560] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] compute.vmdk_allowed_types = ['streamOptimized', 'monolithicSparse'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.555736] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] conductor.workers = 2 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.555913] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] console.allowed_origins = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.556082] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] console.ssl_ciphers = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.556252] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] console.ssl_minimum_version = default {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.556427] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] consoleauth.token_ttl = 600 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.556627] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.cafile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.556783] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.certfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.556944] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.collect_timing = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.557107] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.connect_retries = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.557264] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.connect_retry_delay = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.557414] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.endpoint_override = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.557573] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.insecure = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.557725] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.keyfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.557878] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.max_version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.558044] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.min_version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.558202] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.region_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.558352] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.service_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.558501] nova-conductor[52243]: DEBUG nova.service [None req-fc09e859-22e5-43a9-8e10-6f724bf20701 None None] Creating RPC server for service conductor {{(pid=52243) start /opt/stack/nova/nova/service.py:182}} [ 490.558537] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.service_type = accelerator {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.558662] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.split_loggers = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.558814] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.status_code_retries = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.558969] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.status_code_retry_delay = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.559131] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.timeout = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.559303] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.559457] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] cyborg.version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.559673] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.backend = sqlalchemy {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.560658] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.connection = **** {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.560903] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.connection_debug = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.561605] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.connection_parameters = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.562644] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.connection_recycle_time = 3600 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.562854] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.connection_trace = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.563039] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.db_inc_retry_interval = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.563220] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.db_max_retries = 20 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.563410] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.db_max_retry_interval = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.563574] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.db_retry_interval = 1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.563743] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.max_overflow = 50 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.563901] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.max_pool_size = 5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.564080] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.max_retries = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.564243] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.mysql_enable_ndb = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.564406] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.mysql_sql_mode = TRADITIONAL {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.564565] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.mysql_wsrep_sync_wait = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.564725] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.pool_timeout = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.564887] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.retry_interval = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.565048] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.slave_connection = **** {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.565213] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.sqlite_synchronous = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.565377] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] database.use_db_reconnect = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.565559] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.backend = sqlalchemy {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.565731] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.connection = **** {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.565892] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.connection_debug = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.566239] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.connection_parameters = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.566704] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.connection_recycle_time = 3600 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.566908] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.connection_trace = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.567092] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.db_inc_retry_interval = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.567261] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.db_max_retries = 20 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.567419] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.db_max_retry_interval = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.567865] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.db_retry_interval = 1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.568890] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.max_overflow = 50 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.569673] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.max_pool_size = 5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.571757] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.max_retries = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.571757] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.mysql_enable_ndb = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.571757] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.mysql_sql_mode = TRADITIONAL {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.571757] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.mysql_wsrep_sync_wait = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.571973] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.pool_timeout = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.572129] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.retry_interval = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.572249] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.slave_connection = **** {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.572489] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] api_database.sqlite_synchronous = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.572720] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] devices.enabled_mdev_types = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.572963] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.573207] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ephemeral_storage_encryption.enabled = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.573361] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ephemeral_storage_encryption.key_size = 512 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.573847] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.api_servers = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.575153] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.cafile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.575153] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.certfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.576050] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.collect_timing = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.576396] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.connect_retries = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.577086] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.connect_retry_delay = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.577502] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.debug = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.577822] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.default_trusted_certificate_ids = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.578020] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.enable_certificate_validation = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.578189] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.enable_rbd_download = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.578353] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.endpoint_override = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.578517] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.insecure = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.578776] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.keyfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.578938] nova-conductor[52244]: DEBUG nova.service [None req-29ad9b13-cae7-4199-b8ea-a138d0427279 None None] Join ServiceGroup membership for this service conductor {{(pid=52244) start /opt/stack/nova/nova/service.py:199}} [ 490.579073] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.max_version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.579139] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.min_version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.579188] nova-conductor[52244]: DEBUG nova.servicegroup.drivers.db [None req-29ad9b13-cae7-4199-b8ea-a138d0427279 None None] DB_Driver: join new ServiceGroup member devstack to the conductor group, service = {{(pid=52244) join /opt/stack/nova/nova/servicegroup/drivers/db.py:44}} [ 490.579305] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.num_retries = 3 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.579433] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.rbd_ceph_conf = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.579645] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.rbd_connect_timeout = 5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.579925] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.rbd_pool = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.580110] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.rbd_user = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.580271] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.region_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.580431] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.service_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.580600] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.service_type = image {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.580829] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.split_loggers = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.581708] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.status_code_retries = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.582156] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.status_code_retry_delay = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.582357] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.timeout = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.583422] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.valid_interfaces = ['internal', 'public'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.584063] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.verify_glance_signatures = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.584244] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] glance.version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.584420] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] guestfs.debug = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.584624] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.config_drive_cdrom = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.584785] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.config_drive_inject_password = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.585026] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.dynamic_memory_ratio = 1.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.585379] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.enable_instance_metrics_collection = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.585778] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.enable_remotefx = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.585986] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.instances_path_share = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.586513] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.iscsi_initiator_list = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.586755] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.limit_cpu_features = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.587059] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.mounted_disk_query_retry_count = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.587454] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.587988] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.power_state_check_timeframe = 60 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.588194] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.power_state_event_polling_interval = 2 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.588577] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.589438] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.use_multipath_io = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.589723] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.volume_attach_retry_count = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.589915] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.volume_attach_retry_interval = 5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.590091] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.vswitch_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.590261] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] hyperv.wait_soft_reboot_seconds = 60 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.590659] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] mks.enabled = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.591726] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.591936] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] image_cache.manager_interval = 2400 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.592149] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] image_cache.precache_concurrency = 1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.592325] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] image_cache.remove_unused_base_images = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.592360] nova-conductor[52243]: DEBUG nova.service [None req-fc09e859-22e5-43a9-8e10-6f724bf20701 None None] Join ServiceGroup membership for this service conductor {{(pid=52243) start /opt/stack/nova/nova/service.py:199}} [ 490.592498] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.592579] nova-conductor[52243]: DEBUG nova.servicegroup.drivers.db [None req-fc09e859-22e5-43a9-8e10-6f724bf20701 None None] DB_Driver: join new ServiceGroup member devstack to the conductor group, service = {{(pid=52243) join /opt/stack/nova/nova/servicegroup/drivers/db.py:44}} [ 490.592701] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.592918] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] image_cache.subdirectory_name = _base {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.593138] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.api_max_retries = 60 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.593308] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.api_retry_interval = 2 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.593470] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.auth_section = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.593637] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.auth_type = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.593799] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.cafile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.593954] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.certfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.594128] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.collect_timing = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.594293] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.connect_retries = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.594452] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.connect_retry_delay = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.594607] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.endpoint_override = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.594771] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.insecure = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.594928] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.keyfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.595323] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.max_version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.595890] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.min_version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.596090] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.partition_key = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.596266] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.peer_list = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.596431] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.region_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.596601] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.serial_console_state_timeout = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.596770] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.service_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.596956] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.service_type = baremetal {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.597134] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.split_loggers = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.597292] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.status_code_retries = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.597451] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.status_code_retry_delay = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.597609] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.timeout = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.597786] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.valid_interfaces = ['internal', 'public'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.597942] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ironic.version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.598156] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.598347] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] key_manager.fixed_key = **** {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.598559] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.598742] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican.barbican_api_version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.598899] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican.barbican_endpoint = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.599100] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican.barbican_endpoint_type = public {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.599275] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican.barbican_region_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.599433] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican.cafile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.599607] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican.certfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.599780] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican.collect_timing = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.599941] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican.insecure = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.600108] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican.keyfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.600273] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican.number_of_retries = 60 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.600454] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican.retry_delay = 1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.600633] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican.send_service_user_token = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.600796] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican.split_loggers = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.600951] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican.timeout = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.601129] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican.verify_ssl = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.601285] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican.verify_ssl_path = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.601476] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican_service_user.auth_section = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.601665] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican_service_user.auth_type = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.601828] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican_service_user.cafile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.601981] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican_service_user.certfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.602155] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican_service_user.collect_timing = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.602321] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican_service_user.insecure = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.602472] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican_service_user.keyfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.602630] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican_service_user.split_loggers = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.602780] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] barbican_service_user.timeout = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.602967] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vault.approle_role_id = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.603133] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vault.approle_secret_id = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.603287] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vault.cafile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.603438] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vault.certfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.603596] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vault.collect_timing = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.603753] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vault.insecure = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.603906] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vault.keyfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.604102] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vault.kv_mountpoint = secret {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.604265] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vault.kv_version = 2 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.604426] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vault.namespace = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.604583] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vault.root_token_id = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.604737] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vault.split_loggers = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.604886] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vault.ssl_ca_crt_file = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.605047] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vault.timeout = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.605227] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vault.use_ssl = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.605411] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vault.vault_url = http://127.0.0.1:8200 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.605785] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.cafile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.605785] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.certfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.605924] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.collect_timing = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.606106] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.connect_retries = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.606240] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.connect_retry_delay = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.606399] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.endpoint_override = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.606589] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.insecure = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.606682] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.keyfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.606852] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.max_version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.606974] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.min_version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.607169] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.region_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.607276] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.service_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.607449] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.service_type = identity {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.607589] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.split_loggers = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.607742] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.status_code_retries = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.607894] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.status_code_retry_delay = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.608055] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.timeout = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.608231] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.valid_interfaces = ['internal', 'public'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.608384] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] keystone.version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.608623] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.connection_uri = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.608803] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.cpu_mode = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.609416] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.cpu_model_extra_flags = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.609416] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.cpu_models = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.609416] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.cpu_power_governor_high = performance {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.609538] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.cpu_power_governor_low = powersave {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.609683] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.cpu_power_management = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.609877] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.cpu_power_management_strategy = cpu_state {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.610066] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.device_detach_attempts = 8 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.610323] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.device_detach_timeout = 20 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.610386] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.disk_cachemodes = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.610564] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.disk_prefix = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.610694] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.enabled_perf_events = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.610849] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.file_backed_memory = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.611018] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.gid_maps = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.611191] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.hw_disk_discard = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.611349] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.hw_machine_type = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.611510] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.images_rbd_ceph_conf = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.611694] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.611859] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.612033] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.images_rbd_glance_store_name = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.612201] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.images_rbd_pool = rbd {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.612362] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.images_type = default {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.612516] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.images_volume_group = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.612672] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.inject_key = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.612835] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.inject_partition = -2 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.612991] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.inject_password = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.613182] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.iscsi_iface = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.613508] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.iser_use_multipath = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.613508] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.live_migration_bandwidth = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.613666] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.live_migration_completion_timeout = 800 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.613824] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.live_migration_downtime = 500 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.613977] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.live_migration_downtime_delay = 75 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.614147] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.live_migration_downtime_steps = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.614300] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.live_migration_inbound_addr = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.614454] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.live_migration_permit_auto_converge = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.614609] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.live_migration_permit_post_copy = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.614761] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.live_migration_scheme = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.614927] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.live_migration_timeout_action = abort {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.615093] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.live_migration_tunnelled = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.615249] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.live_migration_uri = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.615406] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.live_migration_with_native_tls = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.615558] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.max_queues = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.615727] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.mem_stats_period_seconds = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.615915] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.nfs_mount_options = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.616266] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.616441] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.num_aoe_discover_tries = 3 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.616603] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.num_iser_scan_tries = 5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.616762] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.num_memory_encrypted_guests = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.616925] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.num_nvme_discover_tries = 5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.617095] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.num_pcie_ports = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.617261] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.num_volume_scan_tries = 5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.617476] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.pmem_namespaces = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.617634] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.quobyte_client_cfg = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.617878] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.618151] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.rbd_connect_timeout = 5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.618311] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.rbd_destroy_volume_retries = 12 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.618469] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.618623] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.rbd_secret_uuid = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.618774] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.rbd_user = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.618931] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.realtime_scheduler_priority = 1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.619207] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.remote_filesystem_transport = ssh {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.619295] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.rescue_image_id = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.619457] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.rescue_kernel_id = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.619645] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.rescue_ramdisk_id = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.619821] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.rng_dev_path = /dev/urandom {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.619983] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.rx_queue_size = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.620162] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.smbfs_mount_options = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.620378] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.620551] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.snapshot_compression = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.620703] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.snapshot_image_format = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.620918] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.621091] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.sparse_logical_volumes = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.621251] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.swtpm_enabled = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.621420] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.swtpm_group = tss {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.621610] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.swtpm_user = tss {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.621787] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.sysinfo_serial = unique {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.621967] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.tx_queue_size = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.622170] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.uid_maps = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.622335] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.use_virtio_for_bridges = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.622502] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.virt_type = kvm {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.622666] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.volume_clear = zero {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.622824] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.volume_clear_size = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.622982] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.volume_use_multipath = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.623148] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.vzstorage_cache_path = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.623312] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.623472] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.vzstorage_mount_group = qemu {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.623633] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.vzstorage_mount_opts = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.623796] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.vzstorage_mount_perms = 0770 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.624023] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.624198] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.vzstorage_mount_user = stack {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.624359] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] libvirt.wait_soft_reboot_seconds = 120 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.624556] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.auth_section = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.624725] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.auth_type = password {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.624885] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.cafile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.625049] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.certfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.625208] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.collect_timing = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.625361] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.connect_retries = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.625512] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.connect_retry_delay = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.625673] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.default_floating_pool = public {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.625825] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.endpoint_override = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.625980] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.extension_sync_interval = 600 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.626147] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.http_retries = 3 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.626303] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.insecure = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.626459] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.keyfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.626614] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.max_version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.626780] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.metadata_proxy_shared_secret = **** {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.626931] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.min_version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.627107] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.ovs_bridge = br-int {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.627268] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.physnets = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.627431] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.region_name = RegionOne {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.627594] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.service_metadata_proxy = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.627749] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.service_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.627917] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.service_type = network {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.628087] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.split_loggers = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.628321] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.status_code_retries = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.628586] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.status_code_retry_delay = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.628786] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.timeout = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.628971] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.valid_interfaces = ['internal', 'public'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.629146] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] neutron.version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.629336] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] notifications.bdms_in_notifications = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.629515] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] notifications.default_level = INFO {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.629706] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] notifications.notification_format = unversioned {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.629870] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] notifications.notify_on_state_change = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.630059] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.630237] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] pci.alias = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.630404] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] pci.device_spec = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.630564] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] pci.report_in_placement = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.630768] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.auth_section = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.630939] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.auth_type = password {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.631120] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.auth_url = http://10.180.1.21/identity {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.631278] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.cafile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.631429] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.certfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.631642] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.collect_timing = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.631810] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.connect_retries = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.631973] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.connect_retry_delay = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.632140] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.default_domain_id = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.632307] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.default_domain_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.632463] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.domain_id = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.632635] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.domain_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.632794] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.endpoint_override = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.632954] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.insecure = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.633119] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.keyfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.633269] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.max_version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.633421] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.min_version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.633584] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.password = **** {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.633737] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.project_domain_id = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.633917] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.project_domain_name = Default {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.634103] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.project_id = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.634284] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.project_name = service {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.634448] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.region_name = RegionOne {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.634601] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.service_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.634764] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.service_type = placement {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.634919] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.split_loggers = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.635091] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.status_code_retries = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.635252] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.status_code_retry_delay = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.635402] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.system_scope = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.635554] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.timeout = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.635707] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.trust_id = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.635859] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.user_domain_id = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.636774] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.user_domain_name = Default {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.636774] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.user_id = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.636774] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.username = placement {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.636774] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.valid_interfaces = ['internal', 'public'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.636774] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] placement.version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.637453] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] quota.cores = 20 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.637453] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] quota.count_usage_from_placement = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.637453] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] quota.driver = nova.quota.DbQuotaDriver {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.637453] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] quota.injected_file_content_bytes = 10240 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.637556] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] quota.injected_file_path_length = 255 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.637674] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] quota.injected_files = 5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.637835] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] quota.instances = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.638048] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] quota.key_pairs = 100 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.638165] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] quota.metadata_items = 128 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.638347] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] quota.ram = 51200 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.638514] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] quota.recheck_quota = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.638639] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] quota.server_group_members = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.638799] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] quota.server_groups = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.638961] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] rdp.enabled = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.639285] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.639503] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.639722] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] scheduler.enable_isolated_aggregate_filtering = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.639920] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] scheduler.image_metadata_prefilter = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.640115] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.640297] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] scheduler.max_attempts = 3 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.640470] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] scheduler.max_placement_results = 1000 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.640647] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] scheduler.placement_aggregate_required_for_tenants = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.640826] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] scheduler.query_placement_for_availability_zone = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.640987] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] scheduler.query_placement_for_image_type_support = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.641178] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.641375] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] scheduler.workers = 2 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.641568] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.641737] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.641945] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.642126] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.642303] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.642478] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.642683] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.642945] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.enabled_filters = ['ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.643158] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.host_subset_size = 1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.643325] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.image_properties_default_architecture = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.643489] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.643654] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.isolated_hosts = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.643840] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.isolated_images = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.644015] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.max_instances_per_host = 50 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.644177] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.max_io_ops_per_host = 8 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.644333] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.pci_in_placement = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.644499] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.644659] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.644818] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.644996] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.645180] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.645339] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.645497] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.track_instance_changes = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.645668] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.645835] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] metrics.required = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.645997] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] metrics.weight_multiplier = 1.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.646171] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] metrics.weight_of_unavailable = -10000.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.646331] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] metrics.weight_setting = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.646639] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.646809] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] serial_console.enabled = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.647009] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] serial_console.port_range = 10000:20000 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.647187] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] serial_console.proxyclient_address = 127.0.0.1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.647353] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] serial_console.serialproxy_host = 0.0.0.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.647519] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] serial_console.serialproxy_port = 6083 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.647682] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] service_user.auth_section = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.647850] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] service_user.auth_type = password {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.648009] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] service_user.cafile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.648168] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] service_user.certfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.648326] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] service_user.collect_timing = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.648482] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] service_user.insecure = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.648634] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] service_user.keyfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.648798] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] service_user.send_service_user_token = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.648955] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] service_user.split_loggers = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.649128] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] service_user.timeout = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.649277] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] spice.agent_enabled = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.649454] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] spice.enabled = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.649818] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.650059] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] spice.html5proxy_host = 0.0.0.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.650231] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] spice.html5proxy_port = 6082 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.650388] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] spice.image_compression = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.650540] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] spice.jpeg_compression = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.650694] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] spice.playback_compression = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.650880] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] spice.server_listen = 127.0.0.1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.651056] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] spice.server_proxyclient_address = 127.0.0.1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.651213] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] spice.streaming_mode = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.651387] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] spice.zlib_compression = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.651562] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] upgrade_levels.baseapi = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.651747] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] upgrade_levels.cert = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.651915] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] upgrade_levels.compute = auto {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.652081] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] upgrade_levels.conductor = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.652236] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] upgrade_levels.scheduler = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.652401] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vendordata_dynamic_auth.auth_section = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.652594] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vendordata_dynamic_auth.auth_type = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.652756] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vendordata_dynamic_auth.cafile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.652907] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vendordata_dynamic_auth.certfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.653078] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vendordata_dynamic_auth.collect_timing = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.653236] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vendordata_dynamic_auth.insecure = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.653389] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vendordata_dynamic_auth.keyfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.653612] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vendordata_dynamic_auth.split_loggers = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.653782] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vendordata_dynamic_auth.timeout = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.653983] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.api_retry_count = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.654165] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.ca_file = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.654344] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.cache_prefix = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.654498] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.cluster_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.654654] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.connection_pool_size = 10 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.654805] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.console_delay_seconds = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.654956] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.datastore_regex = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.655132] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.host_ip = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.655285] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.host_password = **** {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.655443] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.host_port = 443 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.655594] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.host_username = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.655755] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.insecure = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.655911] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.integration_bridge = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.656081] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.maximum_objects = 100 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.656239] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.pbm_default_policy = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.656389] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.pbm_enabled = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.656541] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.pbm_wsdl_location = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.656704] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.656856] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.serial_port_proxy_uri = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.657014] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.serial_port_service_uri = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.657178] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.task_poll_interval = 0.5 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.657334] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.use_linked_clone = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.657494] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.vnc_keymap = en-us {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.657673] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.vnc_port = 5900 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.657832] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vmware.vnc_port_total = 10000 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.658045] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vnc.auth_schemes = ['none'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.658217] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vnc.enabled = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.658518] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.658697] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vnc.novncproxy_host = 0.0.0.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.658862] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vnc.novncproxy_port = 6080 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.659075] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vnc.server_listen = 127.0.0.1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.659220] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.659378] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vnc.vencrypt_ca_certs = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.659539] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vnc.vencrypt_client_cert = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.659717] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] vnc.vencrypt_client_key = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.659923] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.disable_compute_service_check_for_ffu = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.660096] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.disable_deep_image_inspection = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.660255] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.disable_fallback_pcpu_query = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.660414] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.disable_group_policy_check_upcall = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.660866] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.disable_libvirt_livesnapshot = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.660866] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.disable_rootwrap = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.660866] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.enable_numa_live_migration = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.661013] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.enable_qemu_monitor_announce_self = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.661172] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.661325] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.handle_virt_lifecycle_events = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.661574] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.libvirt_disable_apic = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.661732] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.never_download_image_if_on_rbd = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.661899] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.qemu_monitor_announce_self_count = 3 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.662069] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.qemu_monitor_announce_self_interval = 1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.662228] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.662381] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.skip_cpu_compare_at_startup = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.662536] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.skip_cpu_compare_on_dest = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.662696] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.662848] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.skip_reserve_in_use_ironic_nodes = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.663012] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.663186] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.663385] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.663551] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] wsgi.client_socket_timeout = 900 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.663711] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] wsgi.default_pool_size = 1000 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.663887] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] wsgi.keep_alive = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.664085] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] wsgi.max_header_line = 16384 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.664246] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] wsgi.secure_proxy_ssl_header = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.664403] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] wsgi.ssl_ca_file = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.664564] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] wsgi.ssl_cert_file = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.664719] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] wsgi.ssl_key_file = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.664880] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] wsgi.tcp_keepidle = 600 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.665062] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.665227] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] zvm.ca_file = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.665383] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] zvm.cloud_connector_url = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.665600] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.665766] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] zvm.reachable_timeout = 300 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.665986] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_policy.enforce_new_defaults = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.666168] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_policy.enforce_scope = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.666362] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_policy.policy_default_rule = default {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.666555] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_policy.policy_dirs = ['policy.d'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.666746] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_policy.policy_file = policy.yaml {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.666933] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.667120] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.667276] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.667427] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.667586] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.667777] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.667966] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.668183] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] profiler.connection_string = messaging:// {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.668365] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] profiler.enabled = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.668550] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] profiler.es_doc_type = notification {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.668726] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] profiler.es_scroll_size = 10000 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.668888] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] profiler.es_scroll_time = 2m {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.669058] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] profiler.filter_error_trace = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.669221] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] profiler.hmac_keys = SECRET_KEY {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.669378] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] profiler.sentinel_service_name = mymaster {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.669579] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] profiler.socket_timeout = 0.1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.669791] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] profiler.trace_sqlalchemy = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.669989] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] remote_debug.host = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.670165] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] remote_debug.port = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.670350] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.670513] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.670673] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.670851] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.671023] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.671179] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.671339] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.671493] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.671668] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.671824] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.671992] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.672165] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.672330] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.672490] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.672649] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.672819] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.672973] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.673139] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.673299] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.673452] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.673605] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.673765] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.673935] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.674117] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.674281] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.674444] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.ssl = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.674610] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.ssl_ca_file = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.674773] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.ssl_cert_file = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.674927] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.675110] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.ssl_key_file = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.675278] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_rabbit.ssl_version = {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.675480] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.675647] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_notifications.retry = -1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.675850] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.676030] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_messaging_notifications.transport_url = **** {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.676224] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.auth_section = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.676383] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.auth_type = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.676536] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.cafile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.676687] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.certfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.676842] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.collect_timing = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.676993] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.connect_retries = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.677156] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.connect_retry_delay = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.677310] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.endpoint_id = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.677459] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.endpoint_override = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.677614] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.insecure = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.677764] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.keyfile = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.677912] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.max_version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.678074] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.min_version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.678230] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.region_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.678400] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.service_name = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.678551] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.service_type = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.678705] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.split_loggers = False {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.678855] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.status_code_retries = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.679008] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.status_code_retry_delay = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.679166] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.timeout = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.679316] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.valid_interfaces = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.679465] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_limit.version = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.679698] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_reports.file_event_handler = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.679869] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.680033] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] oslo_reports.log_dir = None {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 490.680163] nova-conductor[51822]: DEBUG oslo_service.service [None req-153f317c-afe3-4df9-b7bb-21bb56708cd1 None None] ******************************************************************************** {{(pid=51822) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2613}} [ 577.545547] nova-conductor[52243]: DEBUG oslo_db.sqlalchemy.engines [None req-e4f37662-fb38-469c-9b3d-0645d0530e3c None None] Parent process 51822 forked (52243) with an open database connection, which is being discarded and recreated. {{(pid=52243) checkout /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:434}} [ 618.648149] nova-conductor[52244]: DEBUG oslo_db.sqlalchemy.engines [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Parent process 51822 forked (52244) with an open database connection, which is being discarded and recreated. {{(pid=52244) checkout /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:434}} [ 619.452072] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Took 0.80 seconds to select destinations for 1 instance(s). {{(pid=52244) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 619.481627] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 619.481925] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 619.483645] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 619.491284] nova-conductor[52244]: DEBUG oslo_db.sqlalchemy.engines [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=52244) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 619.571066] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 619.571297] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 619.571783] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 619.572577] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 619.572577] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 619.572577] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 619.584217] nova-conductor[52244]: DEBUG oslo_db.sqlalchemy.engines [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=52244) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 619.606675] nova-conductor[52244]: DEBUG nova.quota [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Getting quotas for project acb2cb42f7814275b0d7369f2e7ab372. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 619.610582] nova-conductor[52244]: DEBUG nova.quota [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Getting quotas for user c807c41f5b8647b982f74a260d5b3c39 and project acb2cb42f7814275b0d7369f2e7ab372. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 619.620453] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] [instance: 42920efb-be41-4813-b33e-d49c6f4fb47c] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52244) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 619.621104] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 619.621319] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 619.621501] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 619.638314] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] [instance: 42920efb-be41-4813-b33e-d49c6f4fb47c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 619.639132] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 619.639450] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 619.639636] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 619.684246] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 619.684634] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 619.684896] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 619.685307] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Acquiring lock "compute-rpcapi-router" {{(pid=52244) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 619.685487] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Acquired lock "compute-rpcapi-router" {{(pid=52244) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 619.686067] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-c0ab1c33-e877-43cd-a9b2-01da01d193e5 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 619.686840] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-c0ab1c33-e877-43cd-a9b2-01da01d193e5 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 619.686840] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-c0ab1c33-e877-43cd-a9b2-01da01d193e5 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 619.687036] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-c0ab1c33-e877-43cd-a9b2-01da01d193e5 None None] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 619.687413] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-c0ab1c33-e877-43cd-a9b2-01da01d193e5 None None] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 619.687611] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-c0ab1c33-e877-43cd-a9b2-01da01d193e5 None None] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 619.696865] nova-conductor[52244]: INFO nova.compute.rpcapi [None req-c0ab1c33-e877-43cd-a9b2-01da01d193e5 None None] Automatically selected compute RPC version 6.2 from minimum service version 66 [ 619.697482] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-c0ab1c33-e877-43cd-a9b2-01da01d193e5 None None] Releasing lock "compute-rpcapi-router" {{(pid=52244) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 619.876775] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Took 0.27 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 619.907609] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 619.908072] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 619.910028] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 619.916446] nova-conductor[52243]: DEBUG oslo_db.sqlalchemy.engines [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=52243) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 619.986069] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 619.986069] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 619.986069] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 619.986069] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 619.986245] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 619.986278] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 619.996026] nova-conductor[52243]: DEBUG oslo_db.sqlalchemy.engines [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=52243) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 620.009592] nova-conductor[52243]: DEBUG nova.quota [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Getting quotas for project fa8a67eb479240bfa25cc71563cdde3d. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 620.013106] nova-conductor[52243]: DEBUG nova.quota [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Getting quotas for user 77b71019b4134229ae7abdc61869bdd6 and project fa8a67eb479240bfa25cc71563cdde3d. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 620.019068] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] [instance: 75398340-5ec7-4e3f-abc1-602f838d7ef3] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 620.019643] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 620.021050] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 620.021050] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 620.025445] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] [instance: 75398340-5ec7-4e3f-abc1-602f838d7ef3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 620.027872] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 620.027872] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 620.027872] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 620.053893] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 620.054145] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 620.054315] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 620.054640] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Acquiring lock "compute-rpcapi-router" {{(pid=52243) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 620.054881] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Acquired lock "compute-rpcapi-router" {{(pid=52243) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 620.055355] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4239bb03-40e4-4d7f-a652-04292bf43205 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 620.055462] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4239bb03-40e4-4d7f-a652-04292bf43205 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 620.055597] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4239bb03-40e4-4d7f-a652-04292bf43205 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 620.056028] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4239bb03-40e4-4d7f-a652-04292bf43205 None None] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 620.056208] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4239bb03-40e4-4d7f-a652-04292bf43205 None None] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 620.056367] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4239bb03-40e4-4d7f-a652-04292bf43205 None None] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 620.062893] nova-conductor[52243]: INFO nova.compute.rpcapi [None req-4239bb03-40e4-4d7f-a652-04292bf43205 None None] Automatically selected compute RPC version 6.2 from minimum service version 66 [ 620.063373] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4239bb03-40e4-4d7f-a652-04292bf43205 None None] Releasing lock "compute-rpcapi-router" {{(pid=52243) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 620.913342] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=52244) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 620.929332] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 620.929631] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 620.929739] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 620.963179] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 620.964230] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 620.964230] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 620.964230] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 620.964230] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 620.964471] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 620.974539] nova-conductor[52244]: DEBUG nova.quota [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Getting quotas for project da38ec4f2cad4c0b8b429771350420a8. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 620.977116] nova-conductor[52244]: DEBUG nova.quota [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Getting quotas for user c8161a460c49451dbc88db22452e20f7 and project da38ec4f2cad4c0b8b429771350420a8. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 620.985330] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] [instance: 5e7637fe-8828-4c16-a629-0d82f1efded9] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52244) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 620.985799] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 620.986054] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 620.987824] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 620.989529] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] [instance: 5e7637fe-8828-4c16-a629-0d82f1efded9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 620.990363] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 620.991366] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 620.991366] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 621.008393] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 621.008393] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 621.008393] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 624.439364] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Took 0.14 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 624.451158] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 624.451394] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 624.451561] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 624.489858] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 624.490245] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 624.490742] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 624.491366] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 624.494030] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 624.494030] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 624.501854] nova-conductor[52243]: DEBUG nova.quota [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Getting quotas for project e35d0d53e9be49c4812e6268e521dfaf. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 624.504701] nova-conductor[52243]: DEBUG nova.quota [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Getting quotas for user ddd7a5cce0914c8cbd4698144cfb5be5 and project e35d0d53e9be49c4812e6268e521dfaf. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 624.511614] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] [instance: 4cdfac2f-6963-4f71-9a42-709f2eeb4f9a] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 624.512552] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 624.512876] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 624.513825] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 624.516337] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] [instance: 4cdfac2f-6963-4f71-9a42-709f2eeb4f9a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 624.516678] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 624.516878] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 624.517062] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 624.533074] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 624.533845] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 624.534094] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 626.141076] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 626.155120] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 626.155510] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 626.155604] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 626.178078] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] [instance: 42920efb-be41-4813-b33e-d49c6f4fb47c] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 8afd2645-2674-4db5-9c6b-4581a3e8bd46, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 42920efb-be41-4813-b33e-d49c6f4fb47c was re-scheduled: Binding failed for port 8afd2645-2674-4db5-9c6b-4581a3e8bd46, please check neutron logs for more information.\n'] [ 626.178461] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 626.178531] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 42920efb-be41-4813-b33e-d49c6f4fb47c.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 42920efb-be41-4813-b33e-d49c6f4fb47c. [ 626.179028] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] [instance: 42920efb-be41-4813-b33e-d49c6f4fb47c] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 42920efb-be41-4813-b33e-d49c6f4fb47c. [ 626.202923] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 626.203160] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 626.203322] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 626.203662] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 626.203874] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 626.203990] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 626.214948] nova-conductor[52243]: DEBUG nova.quota [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Getting quotas for project 6502f31a94194708b7eb572e3c9dd2d6. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 626.217468] nova-conductor[52243]: DEBUG nova.quota [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Getting quotas for user 7e6dba0da3574cf7b9102a17bf8d1a43 and project 6502f31a94194708b7eb572e3c9dd2d6. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 626.222830] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] [instance: b0f907a1-c4f4-4d02-9f07-8a640af4cdc4] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 626.223464] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 626.224807] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 626.224807] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 626.230048] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] [instance: b0f907a1-c4f4-4d02-9f07-8a640af4cdc4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 626.230295] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 626.230966] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 626.230966] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 626.248836] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 626.249168] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 626.249399] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 626.251068] nova-conductor[52244]: DEBUG nova.network.neutron [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] [instance: 42920efb-be41-4813-b33e-d49c6f4fb47c] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 626.773018] nova-conductor[52244]: DEBUG nova.network.neutron [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] [instance: 42920efb-be41-4813-b33e-d49c6f4fb47c] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 626.779498] nova-conductor[52244]: DEBUG nova.network.neutron [None req-79d3ea84-51be-4b75-9270-5e934848994d tempest-ImagesNegativeTestJSON-1415358462 tempest-ImagesNegativeTestJSON-1415358462-project-member] [instance: 42920efb-be41-4813-b33e-d49c6f4fb47c] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 631.150959] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] [instance: 75398340-5ec7-4e3f-abc1-602f838d7ef3] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 7fb0cc84-bd6f-4ed9-a2a5-8e37c68b3ed1, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 75398340-5ec7-4e3f-abc1-602f838d7ef3 was re-scheduled: Binding failed for port 7fb0cc84-bd6f-4ed9-a2a5-8e37c68b3ed1, please check neutron logs for more information.\n'] [ 631.156269] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 631.156646] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 75398340-5ec7-4e3f-abc1-602f838d7ef3.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 75398340-5ec7-4e3f-abc1-602f838d7ef3. [ 631.157500] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] [instance: 75398340-5ec7-4e3f-abc1-602f838d7ef3] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 75398340-5ec7-4e3f-abc1-602f838d7ef3. [ 631.213347] nova-conductor[52243]: DEBUG nova.network.neutron [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] [instance: 75398340-5ec7-4e3f-abc1-602f838d7ef3] deallocate_for_instance() {{(pid=52243) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 631.239493] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Took 0.23 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 631.251408] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 631.251634] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 631.251810] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 631.277779] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 631.277862] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 631.278084] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 631.278334] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 631.279012] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 631.279012] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 631.289522] nova-conductor[52243]: DEBUG nova.quota [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Getting quotas for project c58ea55e246943a083fa2eb0e98cb0c1. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 631.292101] nova-conductor[52243]: DEBUG nova.quota [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Getting quotas for user e12c72b338434c81bc50996b0638cee2 and project c58ea55e246943a083fa2eb0e98cb0c1. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 631.300123] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] [instance: 4a21b251-816d-4668-9a2e-eeabd9ed347b] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 631.300576] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 631.300807] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 631.300982] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 631.303913] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] [instance: 4a21b251-816d-4668-9a2e-eeabd9ed347b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 631.304578] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 631.304991] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 631.304991] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 631.318116] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 631.318344] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 631.318517] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 631.577874] nova-conductor[52243]: DEBUG nova.network.neutron [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] [instance: 75398340-5ec7-4e3f-abc1-602f838d7ef3] Instance cache missing network info. {{(pid=52243) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 631.583304] nova-conductor[52243]: DEBUG nova.network.neutron [None req-ad1b777c-d99f-48ed-9fd1-ccc8fb9bba27 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] [instance: 75398340-5ec7-4e3f-abc1-602f838d7ef3] Updating instance_info_cache with network_info: [] {{(pid=52243) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 632.001360] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] [instance: 5e7637fe-8828-4c16-a629-0d82f1efded9] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port e7aa5e57-f725-4a6c-a6f6-93fb3aad32ea, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 5e7637fe-8828-4c16-a629-0d82f1efded9 was re-scheduled: Binding failed for port e7aa5e57-f725-4a6c-a6f6-93fb3aad32ea, please check neutron logs for more information.\n'] [ 632.002373] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 632.002659] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5e7637fe-8828-4c16-a629-0d82f1efded9.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5e7637fe-8828-4c16-a629-0d82f1efded9. [ 632.003560] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] [instance: 5e7637fe-8828-4c16-a629-0d82f1efded9] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5e7637fe-8828-4c16-a629-0d82f1efded9. [ 632.034283] nova-conductor[52244]: DEBUG nova.network.neutron [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] [instance: 5e7637fe-8828-4c16-a629-0d82f1efded9] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 632.067233] nova-conductor[52244]: DEBUG nova.network.neutron [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] [instance: 5e7637fe-8828-4c16-a629-0d82f1efded9] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 632.072788] nova-conductor[52244]: DEBUG nova.network.neutron [None req-2f705d30-d0d0-4cac-8f43-0110043792d2 tempest-ServerDiagnosticsNegativeTest-1416714911 tempest-ServerDiagnosticsNegativeTest-1416714911-project-member] [instance: 5e7637fe-8828-4c16-a629-0d82f1efded9] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 633.427519] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] [instance: 4cdfac2f-6963-4f71-9a42-709f2eeb4f9a] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 0e4c20cd-dcc1-4b76-a581-ea92335f3e09, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 4cdfac2f-6963-4f71-9a42-709f2eeb4f9a was re-scheduled: Binding failed for port 0e4c20cd-dcc1-4b76-a581-ea92335f3e09, please check neutron logs for more information.\n'] [ 633.431459] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 633.431778] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4cdfac2f-6963-4f71-9a42-709f2eeb4f9a.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4cdfac2f-6963-4f71-9a42-709f2eeb4f9a. [ 633.432010] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] [instance: 4cdfac2f-6963-4f71-9a42-709f2eeb4f9a] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4cdfac2f-6963-4f71-9a42-709f2eeb4f9a. [ 633.465443] nova-conductor[52244]: DEBUG nova.network.neutron [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] [instance: 4cdfac2f-6963-4f71-9a42-709f2eeb4f9a] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 633.506099] nova-conductor[52244]: DEBUG nova.network.neutron [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] [instance: 4cdfac2f-6963-4f71-9a42-709f2eeb4f9a] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 633.514109] nova-conductor[52244]: DEBUG nova.network.neutron [None req-b5ae6b81-1aa9-4a97-9b4a-ac3fc144c56d tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] [instance: 4cdfac2f-6963-4f71-9a42-709f2eeb4f9a] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 633.646537] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Took 0.18 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 633.658507] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 633.658747] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 633.658946] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 633.711506] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 633.711506] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 633.711506] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 633.711506] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 633.711506] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 633.711506] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 633.723134] nova-conductor[52243]: DEBUG nova.quota [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Getting quotas for project 8b11b617cfb243bca8848f333b96f265. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 633.726175] nova-conductor[52243]: DEBUG nova.quota [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Getting quotas for user e4c24dd0950644258b36bb04734df984 and project 8b11b617cfb243bca8848f333b96f265. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 633.734449] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] [instance: dff5937a-0c12-46d4-878a-8c0e783c6695] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 633.735060] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 633.737561] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 633.737561] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 633.739026] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] [instance: dff5937a-0c12-46d4-878a-8c0e783c6695] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 633.739664] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 633.739875] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 633.740098] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 633.758365] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 633.758623] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 633.759867] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 634.617019] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] [instance: b0f907a1-c4f4-4d02-9f07-8a640af4cdc4] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 8ec22c7e-09a3-4c3f-bb3d-32092582ca44, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance b0f907a1-c4f4-4d02-9f07-8a640af4cdc4 was re-scheduled: Binding failed for port 8ec22c7e-09a3-4c3f-bb3d-32092582ca44, please check neutron logs for more information.\n'] [ 634.617019] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 634.617019] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b0f907a1-c4f4-4d02-9f07-8a640af4cdc4.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b0f907a1-c4f4-4d02-9f07-8a640af4cdc4. [ 634.617019] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] [instance: b0f907a1-c4f4-4d02-9f07-8a640af4cdc4] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b0f907a1-c4f4-4d02-9f07-8a640af4cdc4. [ 634.656083] nova-conductor[52243]: DEBUG nova.network.neutron [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] [instance: b0f907a1-c4f4-4d02-9f07-8a640af4cdc4] deallocate_for_instance() {{(pid=52243) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 634.721668] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=52244) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 634.733169] nova-conductor[52243]: DEBUG nova.network.neutron [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] [instance: b0f907a1-c4f4-4d02-9f07-8a640af4cdc4] Instance cache missing network info. {{(pid=52243) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 634.743217] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 634.743217] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 634.743217] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 634.743484] nova-conductor[52243]: DEBUG nova.network.neutron [None req-5f5a6212-82be-4bec-b863-18b40972328e tempest-ServerDiagnosticsTest-1663640938 tempest-ServerDiagnosticsTest-1663640938-project-member] [instance: b0f907a1-c4f4-4d02-9f07-8a640af4cdc4] Updating instance_info_cache with network_info: [] {{(pid=52243) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 634.778489] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 634.778489] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 634.778489] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 634.778757] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 634.779235] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 634.779481] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 634.788564] nova-conductor[52244]: DEBUG nova.quota [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Getting quotas for project dad9bedbe759426d9c98810f2789dbfd. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 634.791178] nova-conductor[52244]: DEBUG nova.quota [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Getting quotas for user 746d830057bd42e49ba0fc3af8a22db6 and project dad9bedbe759426d9c98810f2789dbfd. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 634.798758] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] [instance: 39f071f7-2895-4cf8-aa41-0e683397a2de] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52244) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 634.798888] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 634.799378] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 634.799378] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 634.804343] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] [instance: 39f071f7-2895-4cf8-aa41-0e683397a2de] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 634.805115] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 634.805202] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 634.805368] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 634.817627] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 634.817904] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 634.818122] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 638.282092] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Took 0.15 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 638.296159] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 638.296525] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 638.296741] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 638.330888] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 638.331167] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 638.331414] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 638.331914] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 638.332119] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 638.332286] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 638.345622] nova-conductor[52243]: DEBUG nova.quota [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Getting quotas for project 8c9fe156687d4221b76f0e662bd590a7. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 638.348313] nova-conductor[52243]: DEBUG nova.quota [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Getting quotas for user f1957df6576d4aa19e41c96a046d136e and project 8c9fe156687d4221b76f0e662bd590a7. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 638.360742] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] [instance: 10fc8044-6912-412f-9b84-50efb0e9a398] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 638.361269] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 638.361512] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 638.361645] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 638.364550] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] [instance: 10fc8044-6912-412f-9b84-50efb0e9a398] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 638.365374] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 638.365595] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 638.366069] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 638.386219] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 638.386219] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 638.386219] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 639.700082] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] [instance: dff5937a-0c12-46d4-878a-8c0e783c6695] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port cefa5c6e-e484-4e7d-8eb3-6ba853f51118, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance dff5937a-0c12-46d4-878a-8c0e783c6695 was re-scheduled: Binding failed for port cefa5c6e-e484-4e7d-8eb3-6ba853f51118, please check neutron logs for more information.\n'] [ 639.700668] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 639.700965] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance dff5937a-0c12-46d4-878a-8c0e783c6695.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance dff5937a-0c12-46d4-878a-8c0e783c6695. [ 639.702247] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] [instance: dff5937a-0c12-46d4-878a-8c0e783c6695] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance dff5937a-0c12-46d4-878a-8c0e783c6695. [ 639.738807] nova-conductor[52244]: DEBUG nova.network.neutron [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] [instance: dff5937a-0c12-46d4-878a-8c0e783c6695] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 639.846255] nova-conductor[52244]: DEBUG nova.network.neutron [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] [instance: dff5937a-0c12-46d4-878a-8c0e783c6695] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 639.850638] nova-conductor[52244]: DEBUG nova.network.neutron [None req-ea7ab62e-3009-4f78-a5f1-fb437042f5ef tempest-ServersV294TestFqdnHostnames-1499296313 tempest-ServersV294TestFqdnHostnames-1499296313-project-member] [instance: dff5937a-0c12-46d4-878a-8c0e783c6695] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 640.234318] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] [instance: 4a21b251-816d-4668-9a2e-eeabd9ed347b] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b0fa307d-128a-44a3-990d-452c98019207, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 4a21b251-816d-4668-9a2e-eeabd9ed347b was re-scheduled: Binding failed for port b0fa307d-128a-44a3-990d-452c98019207, please check neutron logs for more information.\n'] [ 640.234927] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 640.235258] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4a21b251-816d-4668-9a2e-eeabd9ed347b.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4a21b251-816d-4668-9a2e-eeabd9ed347b. [ 640.235493] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] [instance: 4a21b251-816d-4668-9a2e-eeabd9ed347b] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4a21b251-816d-4668-9a2e-eeabd9ed347b. [ 640.267018] nova-conductor[52244]: DEBUG nova.network.neutron [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] [instance: 4a21b251-816d-4668-9a2e-eeabd9ed347b] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 640.358750] nova-conductor[52244]: DEBUG nova.network.neutron [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] [instance: 4a21b251-816d-4668-9a2e-eeabd9ed347b] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 640.362865] nova-conductor[52244]: DEBUG nova.network.neutron [None req-e65b6231-b29f-4a1a-bf29-cdf65abe061a tempest-TenantUsagesTestJSON-217578962 tempest-TenantUsagesTestJSON-217578962-project-member] [instance: 4a21b251-816d-4668-9a2e-eeabd9ed347b] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 640.628537] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] [instance: 39f071f7-2895-4cf8-aa41-0e683397a2de] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 64711077-b1f4-4b70-b2fe-7552161f7d16, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 39f071f7-2895-4cf8-aa41-0e683397a2de was re-scheduled: Binding failed for port 64711077-b1f4-4b70-b2fe-7552161f7d16, please check neutron logs for more information.\n'] [ 640.628537] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 640.628537] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 39f071f7-2895-4cf8-aa41-0e683397a2de.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 39f071f7-2895-4cf8-aa41-0e683397a2de. [ 640.628537] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] [instance: 39f071f7-2895-4cf8-aa41-0e683397a2de] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 39f071f7-2895-4cf8-aa41-0e683397a2de. [ 640.653961] nova-conductor[52243]: DEBUG nova.network.neutron [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] [instance: 39f071f7-2895-4cf8-aa41-0e683397a2de] deallocate_for_instance() {{(pid=52243) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 640.703018] nova-conductor[52243]: DEBUG nova.network.neutron [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] [instance: 39f071f7-2895-4cf8-aa41-0e683397a2de] Instance cache missing network info. {{(pid=52243) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 640.708311] nova-conductor[52243]: DEBUG nova.network.neutron [None req-22a13f65-88df-47bd-a90b-d541d8a36016 tempest-ServerExternalEventsTest-103728035 tempest-ServerExternalEventsTest-103728035-project-member] [instance: 39f071f7-2895-4cf8-aa41-0e683397a2de] Updating instance_info_cache with network_info: [] {{(pid=52243) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 642.352623] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Took 0.14 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 642.367445] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 642.367683] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 642.368021] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 642.396710] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 642.396938] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 642.397248] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 642.397466] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 642.397644] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 642.397810] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 642.407368] nova-conductor[52243]: DEBUG nova.quota [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Getting quotas for project e35d0d53e9be49c4812e6268e521dfaf. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 642.409644] nova-conductor[52243]: DEBUG nova.quota [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Getting quotas for user ddd7a5cce0914c8cbd4698144cfb5be5 and project e35d0d53e9be49c4812e6268e521dfaf. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 642.415790] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] [instance: dd588677-08d1-43d8-bff3-62b655a5a194] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 642.416250] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 642.416469] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 642.416636] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 642.419365] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] [instance: dd588677-08d1-43d8-bff3-62b655a5a194] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 642.419990] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 642.420204] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 642.420517] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 642.443886] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 642.443886] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 642.443886] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager [None req-a599293b-9e44-4955-b39b-b79cd6a4522f tempest-AttachInterfacesV270Test-2100975528 tempest-AttachInterfacesV270Test-2100975528-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 646.425399] nova-conductor[52244]: Traceback (most recent call last): [ 646.425399] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 646.425399] nova-conductor[52244]: return func(*args, **kwargs) [ 646.425399] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 646.425399] nova-conductor[52244]: selections = self._select_destinations( [ 646.425399] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 646.425399] nova-conductor[52244]: selections = self._schedule( [ 646.425399] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 646.425399] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 646.425399] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 646.425399] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 646.425399] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager [ 646.425399] nova-conductor[52244]: ERROR nova.conductor.manager [ 646.437505] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a599293b-9e44-4955-b39b-b79cd6a4522f tempest-AttachInterfacesV270Test-2100975528 tempest-AttachInterfacesV270Test-2100975528-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 646.437741] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a599293b-9e44-4955-b39b-b79cd6a4522f tempest-AttachInterfacesV270Test-2100975528 tempest-AttachInterfacesV270Test-2100975528-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 646.440988] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a599293b-9e44-4955-b39b-b79cd6a4522f tempest-AttachInterfacesV270Test-2100975528 tempest-AttachInterfacesV270Test-2100975528-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 646.512377] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-a599293b-9e44-4955-b39b-b79cd6a4522f tempest-AttachInterfacesV270Test-2100975528 tempest-AttachInterfacesV270Test-2100975528-project-member] [instance: a268fbf0-c4dc-42d2-9d49-f8f6c9aee169] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 646.514035] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a599293b-9e44-4955-b39b-b79cd6a4522f tempest-AttachInterfacesV270Test-2100975528 tempest-AttachInterfacesV270Test-2100975528-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 646.514035] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a599293b-9e44-4955-b39b-b79cd6a4522f tempest-AttachInterfacesV270Test-2100975528 tempest-AttachInterfacesV270Test-2100975528-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 646.514035] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a599293b-9e44-4955-b39b-b79cd6a4522f tempest-AttachInterfacesV270Test-2100975528 tempest-AttachInterfacesV270Test-2100975528-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 646.518570] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-a599293b-9e44-4955-b39b-b79cd6a4522f tempest-AttachInterfacesV270Test-2100975528 tempest-AttachInterfacesV270Test-2100975528-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 646.518570] nova-conductor[52244]: Traceback (most recent call last): [ 646.518570] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 646.518570] nova-conductor[52244]: return func(*args, **kwargs) [ 646.518570] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 646.518570] nova-conductor[52244]: selections = self._select_destinations( [ 646.518570] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 646.518570] nova-conductor[52244]: selections = self._schedule( [ 646.518570] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 646.518570] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 646.518570] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 646.518570] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 646.518570] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 646.518570] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 646.519458] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-a599293b-9e44-4955-b39b-b79cd6a4522f tempest-AttachInterfacesV270Test-2100975528 tempest-AttachInterfacesV270Test-2100975528-project-member] [instance: a268fbf0-c4dc-42d2-9d49-f8f6c9aee169] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager [None req-10a7d33e-ba26-496a-a693-f7c1dbfd02d2 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 648.731246] nova-conductor[52243]: Traceback (most recent call last): [ 648.731246] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 648.731246] nova-conductor[52243]: return func(*args, **kwargs) [ 648.731246] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 648.731246] nova-conductor[52243]: selections = self._select_destinations( [ 648.731246] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 648.731246] nova-conductor[52243]: selections = self._schedule( [ 648.731246] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 648.731246] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 648.731246] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 648.731246] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 648.731246] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager [ 648.731246] nova-conductor[52243]: ERROR nova.conductor.manager [ 648.743380] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-10a7d33e-ba26-496a-a693-f7c1dbfd02d2 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 648.743749] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-10a7d33e-ba26-496a-a693-f7c1dbfd02d2 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 648.743944] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-10a7d33e-ba26-496a-a693-f7c1dbfd02d2 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 648.799477] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-10a7d33e-ba26-496a-a693-f7c1dbfd02d2 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] [instance: ce1dc0d1-a5fc-4fe4-b6f1-71d3d5c35a26] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 648.800125] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-10a7d33e-ba26-496a-a693-f7c1dbfd02d2 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 648.800503] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-10a7d33e-ba26-496a-a693-f7c1dbfd02d2 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 648.800503] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-10a7d33e-ba26-496a-a693-f7c1dbfd02d2 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 648.805234] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-10a7d33e-ba26-496a-a693-f7c1dbfd02d2 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 648.805234] nova-conductor[52243]: Traceback (most recent call last): [ 648.805234] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 648.805234] nova-conductor[52243]: return func(*args, **kwargs) [ 648.805234] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 648.805234] nova-conductor[52243]: selections = self._select_destinations( [ 648.805234] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 648.805234] nova-conductor[52243]: selections = self._schedule( [ 648.805234] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 648.805234] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 648.805234] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 648.805234] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 648.805234] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 648.805234] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 648.805900] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-10a7d33e-ba26-496a-a693-f7c1dbfd02d2 tempest-MigrationsAdminTest-1048639197 tempest-MigrationsAdminTest-1048639197-project-member] [instance: ce1dc0d1-a5fc-4fe4-b6f1-71d3d5c35a26] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager [None req-ed784811-79b8-42e8-8ab4-931ac781db0d tempest-ServersAaction247Test-742979346 tempest-ServersAaction247Test-742979346-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 651.710031] nova-conductor[52244]: Traceback (most recent call last): [ 651.710031] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 651.710031] nova-conductor[52244]: return func(*args, **kwargs) [ 651.710031] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 651.710031] nova-conductor[52244]: selections = self._select_destinations( [ 651.710031] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 651.710031] nova-conductor[52244]: selections = self._schedule( [ 651.710031] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 651.710031] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 651.710031] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 651.710031] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 651.710031] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager [ 651.710031] nova-conductor[52244]: ERROR nova.conductor.manager [ 651.720437] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ed784811-79b8-42e8-8ab4-931ac781db0d tempest-ServersAaction247Test-742979346 tempest-ServersAaction247Test-742979346-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 651.720624] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ed784811-79b8-42e8-8ab4-931ac781db0d tempest-ServersAaction247Test-742979346 tempest-ServersAaction247Test-742979346-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 651.720796] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ed784811-79b8-42e8-8ab4-931ac781db0d tempest-ServersAaction247Test-742979346 tempest-ServersAaction247Test-742979346-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 651.774395] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-ed784811-79b8-42e8-8ab4-931ac781db0d tempest-ServersAaction247Test-742979346 tempest-ServersAaction247Test-742979346-project-member] [instance: ede2bc3e-4e4a-43b4-a31c-8d3ff6ccaf0e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 651.775968] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ed784811-79b8-42e8-8ab4-931ac781db0d tempest-ServersAaction247Test-742979346 tempest-ServersAaction247Test-742979346-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 651.775968] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ed784811-79b8-42e8-8ab4-931ac781db0d tempest-ServersAaction247Test-742979346 tempest-ServersAaction247Test-742979346-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 651.775968] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ed784811-79b8-42e8-8ab4-931ac781db0d tempest-ServersAaction247Test-742979346 tempest-ServersAaction247Test-742979346-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 651.779878] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-ed784811-79b8-42e8-8ab4-931ac781db0d tempest-ServersAaction247Test-742979346 tempest-ServersAaction247Test-742979346-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 651.779878] nova-conductor[52244]: Traceback (most recent call last): [ 651.779878] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 651.779878] nova-conductor[52244]: return func(*args, **kwargs) [ 651.779878] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 651.779878] nova-conductor[52244]: selections = self._select_destinations( [ 651.779878] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 651.779878] nova-conductor[52244]: selections = self._schedule( [ 651.779878] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 651.779878] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 651.779878] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 651.779878] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 651.779878] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 651.779878] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 651.780347] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-ed784811-79b8-42e8-8ab4-931ac781db0d tempest-ServersAaction247Test-742979346 tempest-ServersAaction247Test-742979346-project-member] [instance: ede2bc3e-4e4a-43b4-a31c-8d3ff6ccaf0e] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager [None req-66dd64f9-d3a9-468d-b250-58dc7cb11e8e tempest-VolumesAssistedSnapshotsTest-1414147223 tempest-VolumesAssistedSnapshotsTest-1414147223-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 654.084352] nova-conductor[52243]: Traceback (most recent call last): [ 654.084352] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 654.084352] nova-conductor[52243]: return func(*args, **kwargs) [ 654.084352] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 654.084352] nova-conductor[52243]: selections = self._select_destinations( [ 654.084352] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 654.084352] nova-conductor[52243]: selections = self._schedule( [ 654.084352] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 654.084352] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 654.084352] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 654.084352] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 654.084352] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager [ 654.084352] nova-conductor[52243]: ERROR nova.conductor.manager [ 654.092762] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-66dd64f9-d3a9-468d-b250-58dc7cb11e8e tempest-VolumesAssistedSnapshotsTest-1414147223 tempest-VolumesAssistedSnapshotsTest-1414147223-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 654.093045] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-66dd64f9-d3a9-468d-b250-58dc7cb11e8e tempest-VolumesAssistedSnapshotsTest-1414147223 tempest-VolumesAssistedSnapshotsTest-1414147223-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 654.093171] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-66dd64f9-d3a9-468d-b250-58dc7cb11e8e tempest-VolumesAssistedSnapshotsTest-1414147223 tempest-VolumesAssistedSnapshotsTest-1414147223-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 654.154642] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-66dd64f9-d3a9-468d-b250-58dc7cb11e8e tempest-VolumesAssistedSnapshotsTest-1414147223 tempest-VolumesAssistedSnapshotsTest-1414147223-project-member] [instance: 287c0977-e6ad-4704-9f5c-a4fd321c10b0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 654.154642] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-66dd64f9-d3a9-468d-b250-58dc7cb11e8e tempest-VolumesAssistedSnapshotsTest-1414147223 tempest-VolumesAssistedSnapshotsTest-1414147223-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 654.154642] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-66dd64f9-d3a9-468d-b250-58dc7cb11e8e tempest-VolumesAssistedSnapshotsTest-1414147223 tempest-VolumesAssistedSnapshotsTest-1414147223-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 654.154642] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-66dd64f9-d3a9-468d-b250-58dc7cb11e8e tempest-VolumesAssistedSnapshotsTest-1414147223 tempest-VolumesAssistedSnapshotsTest-1414147223-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 654.157142] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-66dd64f9-d3a9-468d-b250-58dc7cb11e8e tempest-VolumesAssistedSnapshotsTest-1414147223 tempest-VolumesAssistedSnapshotsTest-1414147223-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 654.157142] nova-conductor[52243]: Traceback (most recent call last): [ 654.157142] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 654.157142] nova-conductor[52243]: return func(*args, **kwargs) [ 654.157142] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 654.157142] nova-conductor[52243]: selections = self._select_destinations( [ 654.157142] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 654.157142] nova-conductor[52243]: selections = self._schedule( [ 654.157142] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 654.157142] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 654.157142] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 654.157142] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 654.157142] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 654.157142] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 654.158710] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-66dd64f9-d3a9-468d-b250-58dc7cb11e8e tempest-VolumesAssistedSnapshotsTest-1414147223 tempest-VolumesAssistedSnapshotsTest-1414147223-project-member] [instance: 287c0977-e6ad-4704-9f5c-a4fd321c10b0] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager [None req-ff5fa752-c5e5-47ad-88a5-7f8888d27e75 tempest-ServerDiagnosticsV248Test-206544477 tempest-ServerDiagnosticsV248Test-206544477-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 654.721199] nova-conductor[52243]: Traceback (most recent call last): [ 654.721199] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 654.721199] nova-conductor[52243]: return func(*args, **kwargs) [ 654.721199] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 654.721199] nova-conductor[52243]: selections = self._select_destinations( [ 654.721199] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 654.721199] nova-conductor[52243]: selections = self._schedule( [ 654.721199] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 654.721199] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 654.721199] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 654.721199] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 654.721199] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager [ 654.721199] nova-conductor[52243]: ERROR nova.conductor.manager [ 654.730390] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ff5fa752-c5e5-47ad-88a5-7f8888d27e75 tempest-ServerDiagnosticsV248Test-206544477 tempest-ServerDiagnosticsV248Test-206544477-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 654.730831] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ff5fa752-c5e5-47ad-88a5-7f8888d27e75 tempest-ServerDiagnosticsV248Test-206544477 tempest-ServerDiagnosticsV248Test-206544477-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 654.731048] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ff5fa752-c5e5-47ad-88a5-7f8888d27e75 tempest-ServerDiagnosticsV248Test-206544477 tempest-ServerDiagnosticsV248Test-206544477-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 654.811188] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-ff5fa752-c5e5-47ad-88a5-7f8888d27e75 tempest-ServerDiagnosticsV248Test-206544477 tempest-ServerDiagnosticsV248Test-206544477-project-member] [instance: 639a41f1-ec0b-45e7-bf19-5949594ac4c5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 654.811188] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ff5fa752-c5e5-47ad-88a5-7f8888d27e75 tempest-ServerDiagnosticsV248Test-206544477 tempest-ServerDiagnosticsV248Test-206544477-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 654.811188] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ff5fa752-c5e5-47ad-88a5-7f8888d27e75 tempest-ServerDiagnosticsV248Test-206544477 tempest-ServerDiagnosticsV248Test-206544477-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 654.811423] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ff5fa752-c5e5-47ad-88a5-7f8888d27e75 tempest-ServerDiagnosticsV248Test-206544477 tempest-ServerDiagnosticsV248Test-206544477-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 654.816695] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-ff5fa752-c5e5-47ad-88a5-7f8888d27e75 tempest-ServerDiagnosticsV248Test-206544477 tempest-ServerDiagnosticsV248Test-206544477-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 654.816695] nova-conductor[52243]: Traceback (most recent call last): [ 654.816695] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 654.816695] nova-conductor[52243]: return func(*args, **kwargs) [ 654.816695] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 654.816695] nova-conductor[52243]: selections = self._select_destinations( [ 654.816695] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 654.816695] nova-conductor[52243]: selections = self._schedule( [ 654.816695] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 654.816695] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 654.816695] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 654.816695] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 654.816695] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 654.816695] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 654.818314] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-ff5fa752-c5e5-47ad-88a5-7f8888d27e75 tempest-ServerDiagnosticsV248Test-206544477 tempest-ServerDiagnosticsV248Test-206544477-project-member] [instance: 639a41f1-ec0b-45e7-bf19-5949594ac4c5] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 655.681824] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] [instance: dd588677-08d1-43d8-bff3-62b655a5a194] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port d2091aef-34df-49c0-a615-b43ef3305034, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance dd588677-08d1-43d8-bff3-62b655a5a194 was re-scheduled: Binding failed for port d2091aef-34df-49c0-a615-b43ef3305034, please check neutron logs for more information.\n'] [ 655.683246] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 655.683246] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance dd588677-08d1-43d8-bff3-62b655a5a194.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance dd588677-08d1-43d8-bff3-62b655a5a194. [ 655.683246] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] [instance: dd588677-08d1-43d8-bff3-62b655a5a194] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance dd588677-08d1-43d8-bff3-62b655a5a194. [ 655.706734] nova-conductor[52244]: DEBUG nova.network.neutron [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] [instance: dd588677-08d1-43d8-bff3-62b655a5a194] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 655.829976] nova-conductor[52244]: DEBUG nova.network.neutron [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] [instance: dd588677-08d1-43d8-bff3-62b655a5a194] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 655.834063] nova-conductor[52244]: DEBUG nova.network.neutron [None req-056bcff0-52e4-4165-b6bc-eca6ea1b5ae6 tempest-DeleteServersAdminTestJSON-1437841386 tempest-DeleteServersAdminTestJSON-1437841386-project-member] [instance: dd588677-08d1-43d8-bff3-62b655a5a194] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager [None req-1cd2028c-12ad-4bcf-b17d-9844b8fefb93 tempest-ServersWithSpecificFlavorTestJSON-580366979 tempest-ServersWithSpecificFlavorTestJSON-580366979-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 657.816516] nova-conductor[52244]: Traceback (most recent call last): [ 657.816516] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 657.816516] nova-conductor[52244]: return func(*args, **kwargs) [ 657.816516] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 657.816516] nova-conductor[52244]: selections = self._select_destinations( [ 657.816516] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 657.816516] nova-conductor[52244]: selections = self._schedule( [ 657.816516] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 657.816516] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 657.816516] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 657.816516] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 657.816516] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager [ 657.816516] nova-conductor[52244]: ERROR nova.conductor.manager [ 657.851409] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-1cd2028c-12ad-4bcf-b17d-9844b8fefb93 tempest-ServersWithSpecificFlavorTestJSON-580366979 tempest-ServersWithSpecificFlavorTestJSON-580366979-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 657.851683] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-1cd2028c-12ad-4bcf-b17d-9844b8fefb93 tempest-ServersWithSpecificFlavorTestJSON-580366979 tempest-ServersWithSpecificFlavorTestJSON-580366979-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 657.852011] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-1cd2028c-12ad-4bcf-b17d-9844b8fefb93 tempest-ServersWithSpecificFlavorTestJSON-580366979 tempest-ServersWithSpecificFlavorTestJSON-580366979-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 657.910773] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-1cd2028c-12ad-4bcf-b17d-9844b8fefb93 tempest-ServersWithSpecificFlavorTestJSON-580366979 tempest-ServersWithSpecificFlavorTestJSON-580366979-project-member] [instance: de629542-18cb-4cbd-8360-8a6d933c50fa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 657.911770] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-1cd2028c-12ad-4bcf-b17d-9844b8fefb93 tempest-ServersWithSpecificFlavorTestJSON-580366979 tempest-ServersWithSpecificFlavorTestJSON-580366979-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 657.911770] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-1cd2028c-12ad-4bcf-b17d-9844b8fefb93 tempest-ServersWithSpecificFlavorTestJSON-580366979 tempest-ServersWithSpecificFlavorTestJSON-580366979-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 657.911986] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-1cd2028c-12ad-4bcf-b17d-9844b8fefb93 tempest-ServersWithSpecificFlavorTestJSON-580366979 tempest-ServersWithSpecificFlavorTestJSON-580366979-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 657.919434] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-1cd2028c-12ad-4bcf-b17d-9844b8fefb93 tempest-ServersWithSpecificFlavorTestJSON-580366979 tempest-ServersWithSpecificFlavorTestJSON-580366979-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 657.919434] nova-conductor[52244]: Traceback (most recent call last): [ 657.919434] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 657.919434] nova-conductor[52244]: return func(*args, **kwargs) [ 657.919434] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 657.919434] nova-conductor[52244]: selections = self._select_destinations( [ 657.919434] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 657.919434] nova-conductor[52244]: selections = self._schedule( [ 657.919434] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 657.919434] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 657.919434] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 657.919434] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 657.919434] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 657.919434] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 657.920864] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-1cd2028c-12ad-4bcf-b17d-9844b8fefb93 tempest-ServersWithSpecificFlavorTestJSON-580366979 tempest-ServersWithSpecificFlavorTestJSON-580366979-project-member] [instance: de629542-18cb-4cbd-8360-8a6d933c50fa] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 660.186104] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] [instance: 10fc8044-6912-412f-9b84-50efb0e9a398] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port af2b4893-ceaf-45d1-a1df-5e3041a748f9, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 10fc8044-6912-412f-9b84-50efb0e9a398 was re-scheduled: Binding failed for port af2b4893-ceaf-45d1-a1df-5e3041a748f9, please check neutron logs for more information.\n'] [ 660.186681] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 660.187793] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 10fc8044-6912-412f-9b84-50efb0e9a398.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 10fc8044-6912-412f-9b84-50efb0e9a398. [ 660.187883] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] [instance: 10fc8044-6912-412f-9b84-50efb0e9a398] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 10fc8044-6912-412f-9b84-50efb0e9a398. [ 660.217407] nova-conductor[52244]: DEBUG nova.network.neutron [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] [instance: 10fc8044-6912-412f-9b84-50efb0e9a398] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 660.326562] nova-conductor[52244]: DEBUG nova.network.neutron [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] [instance: 10fc8044-6912-412f-9b84-50efb0e9a398] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 660.331387] nova-conductor[52244]: DEBUG nova.network.neutron [None req-afee1a71-dd09-4d66-8b96-5a4780ed3593 tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] [instance: 10fc8044-6912-412f-9b84-50efb0e9a398] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager [None req-87e5cdff-76b6-4045-9c9b-a5b164abff38 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 660.688145] nova-conductor[52244]: Traceback (most recent call last): [ 660.688145] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 660.688145] nova-conductor[52244]: return func(*args, **kwargs) [ 660.688145] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 660.688145] nova-conductor[52244]: selections = self._select_destinations( [ 660.688145] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 660.688145] nova-conductor[52244]: selections = self._schedule( [ 660.688145] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 660.688145] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 660.688145] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 660.688145] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 660.688145] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager [ 660.688145] nova-conductor[52244]: ERROR nova.conductor.manager [ 660.694922] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-87e5cdff-76b6-4045-9c9b-a5b164abff38 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 660.695319] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-87e5cdff-76b6-4045-9c9b-a5b164abff38 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 660.695721] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-87e5cdff-76b6-4045-9c9b-a5b164abff38 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 660.767027] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-87e5cdff-76b6-4045-9c9b-a5b164abff38 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] [instance: b09fba68-8f40-473c-a1b8-d58e494e79b6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 660.767027] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-87e5cdff-76b6-4045-9c9b-a5b164abff38 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 660.767027] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-87e5cdff-76b6-4045-9c9b-a5b164abff38 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 660.767027] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-87e5cdff-76b6-4045-9c9b-a5b164abff38 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 660.771913] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-87e5cdff-76b6-4045-9c9b-a5b164abff38 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 660.771913] nova-conductor[52244]: Traceback (most recent call last): [ 660.771913] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 660.771913] nova-conductor[52244]: return func(*args, **kwargs) [ 660.771913] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 660.771913] nova-conductor[52244]: selections = self._select_destinations( [ 660.771913] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 660.771913] nova-conductor[52244]: selections = self._schedule( [ 660.771913] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 660.771913] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 660.771913] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 660.771913] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 660.771913] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 660.771913] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 660.775024] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-87e5cdff-76b6-4045-9c9b-a5b164abff38 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] [instance: b09fba68-8f40-473c-a1b8-d58e494e79b6] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager [None req-cf55f77b-3a11-4db5-a587-c9eed79374fc tempest-AttachInterfacesUnderV243Test-1875895513 tempest-AttachInterfacesUnderV243Test-1875895513-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 664.428228] nova-conductor[52243]: Traceback (most recent call last): [ 664.428228] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 664.428228] nova-conductor[52243]: return func(*args, **kwargs) [ 664.428228] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 664.428228] nova-conductor[52243]: selections = self._select_destinations( [ 664.428228] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 664.428228] nova-conductor[52243]: selections = self._schedule( [ 664.428228] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 664.428228] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 664.428228] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 664.428228] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 664.428228] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager [ 664.428228] nova-conductor[52243]: ERROR nova.conductor.manager [ 664.438924] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-cf55f77b-3a11-4db5-a587-c9eed79374fc tempest-AttachInterfacesUnderV243Test-1875895513 tempest-AttachInterfacesUnderV243Test-1875895513-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 664.444878] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-cf55f77b-3a11-4db5-a587-c9eed79374fc tempest-AttachInterfacesUnderV243Test-1875895513 tempest-AttachInterfacesUnderV243Test-1875895513-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 664.444878] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-cf55f77b-3a11-4db5-a587-c9eed79374fc tempest-AttachInterfacesUnderV243Test-1875895513 tempest-AttachInterfacesUnderV243Test-1875895513-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 664.504185] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-cf55f77b-3a11-4db5-a587-c9eed79374fc tempest-AttachInterfacesUnderV243Test-1875895513 tempest-AttachInterfacesUnderV243Test-1875895513-project-member] [instance: d8986ce9-b41e-47c0-8dd8-7643a97f7fd1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 664.504923] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-cf55f77b-3a11-4db5-a587-c9eed79374fc tempest-AttachInterfacesUnderV243Test-1875895513 tempest-AttachInterfacesUnderV243Test-1875895513-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 664.505147] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-cf55f77b-3a11-4db5-a587-c9eed79374fc tempest-AttachInterfacesUnderV243Test-1875895513 tempest-AttachInterfacesUnderV243Test-1875895513-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 664.505382] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-cf55f77b-3a11-4db5-a587-c9eed79374fc tempest-AttachInterfacesUnderV243Test-1875895513 tempest-AttachInterfacesUnderV243Test-1875895513-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 664.508538] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-cf55f77b-3a11-4db5-a587-c9eed79374fc tempest-AttachInterfacesUnderV243Test-1875895513 tempest-AttachInterfacesUnderV243Test-1875895513-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 664.508538] nova-conductor[52243]: Traceback (most recent call last): [ 664.508538] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 664.508538] nova-conductor[52243]: return func(*args, **kwargs) [ 664.508538] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 664.508538] nova-conductor[52243]: selections = self._select_destinations( [ 664.508538] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 664.508538] nova-conductor[52243]: selections = self._schedule( [ 664.508538] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 664.508538] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 664.508538] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 664.508538] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 664.508538] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 664.508538] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 664.509079] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-cf55f77b-3a11-4db5-a587-c9eed79374fc tempest-AttachInterfacesUnderV243Test-1875895513 tempest-AttachInterfacesUnderV243Test-1875895513-project-member] [instance: d8986ce9-b41e-47c0-8dd8-7643a97f7fd1] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager [None req-ea878fc6-1b68-45dd-9cf9-86e0eeb13b08 tempest-ServersAdmin275Test-1034469438 tempest-ServersAdmin275Test-1034469438-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 664.608745] nova-conductor[52244]: Traceback (most recent call last): [ 664.608745] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 664.608745] nova-conductor[52244]: return func(*args, **kwargs) [ 664.608745] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 664.608745] nova-conductor[52244]: selections = self._select_destinations( [ 664.608745] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 664.608745] nova-conductor[52244]: selections = self._schedule( [ 664.608745] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 664.608745] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 664.608745] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 664.608745] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 664.608745] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager [ 664.608745] nova-conductor[52244]: ERROR nova.conductor.manager [ 664.619091] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ea878fc6-1b68-45dd-9cf9-86e0eeb13b08 tempest-ServersAdmin275Test-1034469438 tempest-ServersAdmin275Test-1034469438-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 664.619392] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ea878fc6-1b68-45dd-9cf9-86e0eeb13b08 tempest-ServersAdmin275Test-1034469438 tempest-ServersAdmin275Test-1034469438-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 664.619603] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ea878fc6-1b68-45dd-9cf9-86e0eeb13b08 tempest-ServersAdmin275Test-1034469438 tempest-ServersAdmin275Test-1034469438-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 664.694366] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-ea878fc6-1b68-45dd-9cf9-86e0eeb13b08 tempest-ServersAdmin275Test-1034469438 tempest-ServersAdmin275Test-1034469438-project-member] [instance: 68ea9b7a-7fda-49ab-b8d3-ac42feb181aa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 664.699130] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ea878fc6-1b68-45dd-9cf9-86e0eeb13b08 tempest-ServersAdmin275Test-1034469438 tempest-ServersAdmin275Test-1034469438-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 664.699865] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ea878fc6-1b68-45dd-9cf9-86e0eeb13b08 tempest-ServersAdmin275Test-1034469438 tempest-ServersAdmin275Test-1034469438-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 664.699865] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ea878fc6-1b68-45dd-9cf9-86e0eeb13b08 tempest-ServersAdmin275Test-1034469438 tempest-ServersAdmin275Test-1034469438-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 664.704317] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-ea878fc6-1b68-45dd-9cf9-86e0eeb13b08 tempest-ServersAdmin275Test-1034469438 tempest-ServersAdmin275Test-1034469438-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 664.704317] nova-conductor[52244]: Traceback (most recent call last): [ 664.704317] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 664.704317] nova-conductor[52244]: return func(*args, **kwargs) [ 664.704317] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 664.704317] nova-conductor[52244]: selections = self._select_destinations( [ 664.704317] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 664.704317] nova-conductor[52244]: selections = self._schedule( [ 664.704317] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 664.704317] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 664.704317] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 664.704317] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 664.704317] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 664.704317] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 664.705508] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-ea878fc6-1b68-45dd-9cf9-86e0eeb13b08 tempest-ServersAdmin275Test-1034469438 tempest-ServersAdmin275Test-1034469438-project-member] [instance: 68ea9b7a-7fda-49ab-b8d3-ac42feb181aa] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 676.926515] nova-conductor[52243]: Traceback (most recent call last): [ 676.926515] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 676.926515] nova-conductor[52243]: return func(*args, **kwargs) [ 676.926515] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 676.926515] nova-conductor[52243]: selections = self._select_destinations( [ 676.926515] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 676.926515] nova-conductor[52243]: selections = self._schedule( [ 676.926515] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 676.926515] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 676.926515] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 676.926515] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 676.926515] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager [ 676.926515] nova-conductor[52243]: ERROR nova.conductor.manager [ 676.940287] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 676.940394] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 676.940616] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 676.996209] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] [instance: 077e7b18-cc79-46e5-9347-ec5847df3220] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 676.996923] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 676.997567] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 676.997567] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.001681] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 677.001681] nova-conductor[52243]: Traceback (most recent call last): [ 677.001681] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 677.001681] nova-conductor[52243]: return func(*args, **kwargs) [ 677.001681] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 677.001681] nova-conductor[52243]: selections = self._select_destinations( [ 677.001681] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 677.001681] nova-conductor[52243]: selections = self._schedule( [ 677.001681] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 677.001681] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 677.001681] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 677.001681] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 677.001681] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 677.001681] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 677.002363] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] [instance: 077e7b18-cc79-46e5-9347-ec5847df3220] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 677.033299] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.033538] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.034212] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.080098] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] [instance: 103d252a-4723-4f02-9c8b-77d48284705c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 677.080437] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.080675] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.080795] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.084373] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 677.084373] nova-conductor[52243]: Traceback (most recent call last): [ 677.084373] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 677.084373] nova-conductor[52243]: return func(*args, **kwargs) [ 677.084373] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 677.084373] nova-conductor[52243]: selections = self._select_destinations( [ 677.084373] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 677.084373] nova-conductor[52243]: selections = self._schedule( [ 677.084373] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 677.084373] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 677.084373] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 677.084373] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 677.084373] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 677.084373] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 677.084841] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] [instance: 103d252a-4723-4f02-9c8b-77d48284705c] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 677.116986] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.116986] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.116986] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.177552] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] [instance: 711970b0-1099-4e30-90a6-1df1b5450713] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 677.181020] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.181020] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.181020] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.183809] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 677.183809] nova-conductor[52243]: Traceback (most recent call last): [ 677.183809] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 677.183809] nova-conductor[52243]: return func(*args, **kwargs) [ 677.183809] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 677.183809] nova-conductor[52243]: selections = self._select_destinations( [ 677.183809] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 677.183809] nova-conductor[52243]: selections = self._schedule( [ 677.183809] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 677.183809] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 677.183809] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 677.183809] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 677.183809] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 677.183809] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 677.184381] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-ddd2e828-7280-4440-8cc5-1ec91cd7944e tempest-ListServersNegativeTestJSON-1833845881 tempest-ListServersNegativeTestJSON-1833845881-project-member] [instance: 711970b0-1099-4e30-90a6-1df1b5450713] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager [None req-891c5445-b7eb-4a94-8e0d-c677614054cc tempest-FloatingIPsAssociationNegativeTestJSON-1669177020 tempest-FloatingIPsAssociationNegativeTestJSON-1669177020-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 677.564085] nova-conductor[52244]: Traceback (most recent call last): [ 677.564085] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 677.564085] nova-conductor[52244]: return func(*args, **kwargs) [ 677.564085] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 677.564085] nova-conductor[52244]: selections = self._select_destinations( [ 677.564085] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 677.564085] nova-conductor[52244]: selections = self._schedule( [ 677.564085] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 677.564085] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 677.564085] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 677.564085] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 677.564085] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager [ 677.564085] nova-conductor[52244]: ERROR nova.conductor.manager [ 677.571913] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-891c5445-b7eb-4a94-8e0d-c677614054cc tempest-FloatingIPsAssociationNegativeTestJSON-1669177020 tempest-FloatingIPsAssociationNegativeTestJSON-1669177020-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.572108] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-891c5445-b7eb-4a94-8e0d-c677614054cc tempest-FloatingIPsAssociationNegativeTestJSON-1669177020 tempest-FloatingIPsAssociationNegativeTestJSON-1669177020-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.572224] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-891c5445-b7eb-4a94-8e0d-c677614054cc tempest-FloatingIPsAssociationNegativeTestJSON-1669177020 tempest-FloatingIPsAssociationNegativeTestJSON-1669177020-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.630015] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-891c5445-b7eb-4a94-8e0d-c677614054cc tempest-FloatingIPsAssociationNegativeTestJSON-1669177020 tempest-FloatingIPsAssociationNegativeTestJSON-1669177020-project-member] [instance: d9d81e57-4969-4fbf-b49e-7d0fe447e91b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 677.630833] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-891c5445-b7eb-4a94-8e0d-c677614054cc tempest-FloatingIPsAssociationNegativeTestJSON-1669177020 tempest-FloatingIPsAssociationNegativeTestJSON-1669177020-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.631771] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-891c5445-b7eb-4a94-8e0d-c677614054cc tempest-FloatingIPsAssociationNegativeTestJSON-1669177020 tempest-FloatingIPsAssociationNegativeTestJSON-1669177020-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.631771] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-891c5445-b7eb-4a94-8e0d-c677614054cc tempest-FloatingIPsAssociationNegativeTestJSON-1669177020 tempest-FloatingIPsAssociationNegativeTestJSON-1669177020-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.635506] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-891c5445-b7eb-4a94-8e0d-c677614054cc tempest-FloatingIPsAssociationNegativeTestJSON-1669177020 tempest-FloatingIPsAssociationNegativeTestJSON-1669177020-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 677.635506] nova-conductor[52244]: Traceback (most recent call last): [ 677.635506] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 677.635506] nova-conductor[52244]: return func(*args, **kwargs) [ 677.635506] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 677.635506] nova-conductor[52244]: selections = self._select_destinations( [ 677.635506] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 677.635506] nova-conductor[52244]: selections = self._schedule( [ 677.635506] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 677.635506] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 677.635506] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 677.635506] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 677.635506] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 677.635506] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 677.636122] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-891c5445-b7eb-4a94-8e0d-c677614054cc tempest-FloatingIPsAssociationNegativeTestJSON-1669177020 tempest-FloatingIPsAssociationNegativeTestJSON-1669177020-project-member] [instance: d9d81e57-4969-4fbf-b49e-7d0fe447e91b] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager [None req-3831ef87-c4b3-4d6c-9622-6916b6b2bb67 tempest-InstanceActionsTestJSON-50949956 tempest-InstanceActionsTestJSON-50949956-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 678.836796] nova-conductor[52243]: Traceback (most recent call last): [ 678.836796] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 678.836796] nova-conductor[52243]: return func(*args, **kwargs) [ 678.836796] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 678.836796] nova-conductor[52243]: selections = self._select_destinations( [ 678.836796] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 678.836796] nova-conductor[52243]: selections = self._schedule( [ 678.836796] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 678.836796] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 678.836796] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 678.836796] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 678.836796] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager [ 678.836796] nova-conductor[52243]: ERROR nova.conductor.manager [ 678.849307] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-3831ef87-c4b3-4d6c-9622-6916b6b2bb67 tempest-InstanceActionsTestJSON-50949956 tempest-InstanceActionsTestJSON-50949956-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 678.849630] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-3831ef87-c4b3-4d6c-9622-6916b6b2bb67 tempest-InstanceActionsTestJSON-50949956 tempest-InstanceActionsTestJSON-50949956-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 678.849860] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-3831ef87-c4b3-4d6c-9622-6916b6b2bb67 tempest-InstanceActionsTestJSON-50949956 tempest-InstanceActionsTestJSON-50949956-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 678.921951] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-3831ef87-c4b3-4d6c-9622-6916b6b2bb67 tempest-InstanceActionsTestJSON-50949956 tempest-InstanceActionsTestJSON-50949956-project-member] [instance: 8b238463-aa3f-4ded-aac8-f0f64a142be9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 678.921951] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-3831ef87-c4b3-4d6c-9622-6916b6b2bb67 tempest-InstanceActionsTestJSON-50949956 tempest-InstanceActionsTestJSON-50949956-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 678.921951] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-3831ef87-c4b3-4d6c-9622-6916b6b2bb67 tempest-InstanceActionsTestJSON-50949956 tempest-InstanceActionsTestJSON-50949956-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 678.921951] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-3831ef87-c4b3-4d6c-9622-6916b6b2bb67 tempest-InstanceActionsTestJSON-50949956 tempest-InstanceActionsTestJSON-50949956-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 678.926093] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-3831ef87-c4b3-4d6c-9622-6916b6b2bb67 tempest-InstanceActionsTestJSON-50949956 tempest-InstanceActionsTestJSON-50949956-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 678.926093] nova-conductor[52243]: Traceback (most recent call last): [ 678.926093] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 678.926093] nova-conductor[52243]: return func(*args, **kwargs) [ 678.926093] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 678.926093] nova-conductor[52243]: selections = self._select_destinations( [ 678.926093] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 678.926093] nova-conductor[52243]: selections = self._schedule( [ 678.926093] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 678.926093] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 678.926093] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 678.926093] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 678.926093] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 678.926093] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 678.926729] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-3831ef87-c4b3-4d6c-9622-6916b6b2bb67 tempest-InstanceActionsTestJSON-50949956 tempest-InstanceActionsTestJSON-50949956-project-member] [instance: 8b238463-aa3f-4ded-aac8-f0f64a142be9] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager [None req-6ebd77c2-5c23-4f3e-801a-62623ded21d9 tempest-ServerTagsTestJSON-94776460 tempest-ServerTagsTestJSON-94776460-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 679.971269] nova-conductor[52244]: Traceback (most recent call last): [ 679.971269] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 679.971269] nova-conductor[52244]: return func(*args, **kwargs) [ 679.971269] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 679.971269] nova-conductor[52244]: selections = self._select_destinations( [ 679.971269] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 679.971269] nova-conductor[52244]: selections = self._schedule( [ 679.971269] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 679.971269] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 679.971269] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 679.971269] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 679.971269] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager [ 679.971269] nova-conductor[52244]: ERROR nova.conductor.manager [ 679.981066] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6ebd77c2-5c23-4f3e-801a-62623ded21d9 tempest-ServerTagsTestJSON-94776460 tempest-ServerTagsTestJSON-94776460-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 679.981066] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6ebd77c2-5c23-4f3e-801a-62623ded21d9 tempest-ServerTagsTestJSON-94776460 tempest-ServerTagsTestJSON-94776460-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 679.981066] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6ebd77c2-5c23-4f3e-801a-62623ded21d9 tempest-ServerTagsTestJSON-94776460 tempest-ServerTagsTestJSON-94776460-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 680.032480] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-6ebd77c2-5c23-4f3e-801a-62623ded21d9 tempest-ServerTagsTestJSON-94776460 tempest-ServerTagsTestJSON-94776460-project-member] [instance: 65c88d52-d555-4b66-b11a-b59e44380632] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 680.034617] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6ebd77c2-5c23-4f3e-801a-62623ded21d9 tempest-ServerTagsTestJSON-94776460 tempest-ServerTagsTestJSON-94776460-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 680.034617] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6ebd77c2-5c23-4f3e-801a-62623ded21d9 tempest-ServerTagsTestJSON-94776460 tempest-ServerTagsTestJSON-94776460-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 680.034617] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6ebd77c2-5c23-4f3e-801a-62623ded21d9 tempest-ServerTagsTestJSON-94776460 tempest-ServerTagsTestJSON-94776460-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 680.037948] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-6ebd77c2-5c23-4f3e-801a-62623ded21d9 tempest-ServerTagsTestJSON-94776460 tempest-ServerTagsTestJSON-94776460-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 680.037948] nova-conductor[52244]: Traceback (most recent call last): [ 680.037948] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 680.037948] nova-conductor[52244]: return func(*args, **kwargs) [ 680.037948] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 680.037948] nova-conductor[52244]: selections = self._select_destinations( [ 680.037948] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 680.037948] nova-conductor[52244]: selections = self._schedule( [ 680.037948] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 680.037948] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 680.037948] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 680.037948] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 680.037948] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 680.037948] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 680.038570] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-6ebd77c2-5c23-4f3e-801a-62623ded21d9 tempest-ServerTagsTestJSON-94776460 tempest-ServerTagsTestJSON-94776460-project-member] [instance: 65c88d52-d555-4b66-b11a-b59e44380632] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager [None req-1f14ae9a-f612-4e14-8226-b51c91f10dc2 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 681.589439] nova-conductor[52243]: Traceback (most recent call last): [ 681.589439] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 681.589439] nova-conductor[52243]: return func(*args, **kwargs) [ 681.589439] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 681.589439] nova-conductor[52243]: selections = self._select_destinations( [ 681.589439] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 681.589439] nova-conductor[52243]: selections = self._schedule( [ 681.589439] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 681.589439] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 681.589439] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 681.589439] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 681.589439] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager [ 681.589439] nova-conductor[52243]: ERROR nova.conductor.manager [ 681.604041] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-1f14ae9a-f612-4e14-8226-b51c91f10dc2 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 681.604041] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-1f14ae9a-f612-4e14-8226-b51c91f10dc2 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 681.604041] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-1f14ae9a-f612-4e14-8226-b51c91f10dc2 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 681.655334] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-1f14ae9a-f612-4e14-8226-b51c91f10dc2 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] [instance: 93c7d003-8b18-45b6-97db-fea68c5f6fab] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 681.656083] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-1f14ae9a-f612-4e14-8226-b51c91f10dc2 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 681.656325] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-1f14ae9a-f612-4e14-8226-b51c91f10dc2 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 681.656501] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-1f14ae9a-f612-4e14-8226-b51c91f10dc2 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 681.659644] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-1f14ae9a-f612-4e14-8226-b51c91f10dc2 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 681.659644] nova-conductor[52243]: Traceback (most recent call last): [ 681.659644] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 681.659644] nova-conductor[52243]: return func(*args, **kwargs) [ 681.659644] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 681.659644] nova-conductor[52243]: selections = self._select_destinations( [ 681.659644] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 681.659644] nova-conductor[52243]: selections = self._schedule( [ 681.659644] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 681.659644] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 681.659644] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 681.659644] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 681.659644] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 681.659644] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 681.660512] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-1f14ae9a-f612-4e14-8226-b51c91f10dc2 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] [instance: 93c7d003-8b18-45b6-97db-fea68c5f6fab] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager [None req-2c819f17-b05a-4752-8a36-6bd4c7efeceb tempest-ImagesOneServerNegativeTestJSON-728750141 tempest-ImagesOneServerNegativeTestJSON-728750141-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 683.155958] nova-conductor[52244]: Traceback (most recent call last): [ 683.155958] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 683.155958] nova-conductor[52244]: return func(*args, **kwargs) [ 683.155958] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 683.155958] nova-conductor[52244]: selections = self._select_destinations( [ 683.155958] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 683.155958] nova-conductor[52244]: selections = self._schedule( [ 683.155958] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 683.155958] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 683.155958] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 683.155958] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 683.155958] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager [ 683.155958] nova-conductor[52244]: ERROR nova.conductor.manager [ 683.168173] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2c819f17-b05a-4752-8a36-6bd4c7efeceb tempest-ImagesOneServerNegativeTestJSON-728750141 tempest-ImagesOneServerNegativeTestJSON-728750141-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 683.168414] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2c819f17-b05a-4752-8a36-6bd4c7efeceb tempest-ImagesOneServerNegativeTestJSON-728750141 tempest-ImagesOneServerNegativeTestJSON-728750141-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 683.168832] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2c819f17-b05a-4752-8a36-6bd4c7efeceb tempest-ImagesOneServerNegativeTestJSON-728750141 tempest-ImagesOneServerNegativeTestJSON-728750141-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 683.238642] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-2c819f17-b05a-4752-8a36-6bd4c7efeceb tempest-ImagesOneServerNegativeTestJSON-728750141 tempest-ImagesOneServerNegativeTestJSON-728750141-project-member] [instance: 6b473921-600d-4d9b-95d2-f7552026dd0b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 683.239409] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2c819f17-b05a-4752-8a36-6bd4c7efeceb tempest-ImagesOneServerNegativeTestJSON-728750141 tempest-ImagesOneServerNegativeTestJSON-728750141-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 683.239623] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2c819f17-b05a-4752-8a36-6bd4c7efeceb tempest-ImagesOneServerNegativeTestJSON-728750141 tempest-ImagesOneServerNegativeTestJSON-728750141-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 683.239781] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2c819f17-b05a-4752-8a36-6bd4c7efeceb tempest-ImagesOneServerNegativeTestJSON-728750141 tempest-ImagesOneServerNegativeTestJSON-728750141-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 683.244720] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-2c819f17-b05a-4752-8a36-6bd4c7efeceb tempest-ImagesOneServerNegativeTestJSON-728750141 tempest-ImagesOneServerNegativeTestJSON-728750141-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 683.244720] nova-conductor[52244]: Traceback (most recent call last): [ 683.244720] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 683.244720] nova-conductor[52244]: return func(*args, **kwargs) [ 683.244720] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 683.244720] nova-conductor[52244]: selections = self._select_destinations( [ 683.244720] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 683.244720] nova-conductor[52244]: selections = self._schedule( [ 683.244720] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 683.244720] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 683.244720] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 683.244720] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 683.244720] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 683.244720] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 683.245297] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-2c819f17-b05a-4752-8a36-6bd4c7efeceb tempest-ImagesOneServerNegativeTestJSON-728750141 tempest-ImagesOneServerNegativeTestJSON-728750141-project-member] [instance: 6b473921-600d-4d9b-95d2-f7552026dd0b] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager [None req-f7bee0b7-3179-433d-8ad1-fa7369dca9b3 tempest-ServerRescueTestJSON-2068892925 tempest-ServerRescueTestJSON-2068892925-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 684.532453] nova-conductor[52243]: Traceback (most recent call last): [ 684.532453] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 684.532453] nova-conductor[52243]: return func(*args, **kwargs) [ 684.532453] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 684.532453] nova-conductor[52243]: selections = self._select_destinations( [ 684.532453] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 684.532453] nova-conductor[52243]: selections = self._schedule( [ 684.532453] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 684.532453] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 684.532453] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 684.532453] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 684.532453] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager [ 684.532453] nova-conductor[52243]: ERROR nova.conductor.manager [ 684.543686] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-f7bee0b7-3179-433d-8ad1-fa7369dca9b3 tempest-ServerRescueTestJSON-2068892925 tempest-ServerRescueTestJSON-2068892925-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 684.543686] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-f7bee0b7-3179-433d-8ad1-fa7369dca9b3 tempest-ServerRescueTestJSON-2068892925 tempest-ServerRescueTestJSON-2068892925-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 684.543960] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-f7bee0b7-3179-433d-8ad1-fa7369dca9b3 tempest-ServerRescueTestJSON-2068892925 tempest-ServerRescueTestJSON-2068892925-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 684.591892] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-f7bee0b7-3179-433d-8ad1-fa7369dca9b3 tempest-ServerRescueTestJSON-2068892925 tempest-ServerRescueTestJSON-2068892925-project-member] [instance: 0e7d3589-5c00-43b0-b3af-920958d1d6c1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 684.592677] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-f7bee0b7-3179-433d-8ad1-fa7369dca9b3 tempest-ServerRescueTestJSON-2068892925 tempest-ServerRescueTestJSON-2068892925-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 684.593372] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-f7bee0b7-3179-433d-8ad1-fa7369dca9b3 tempest-ServerRescueTestJSON-2068892925 tempest-ServerRescueTestJSON-2068892925-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 684.593372] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-f7bee0b7-3179-433d-8ad1-fa7369dca9b3 tempest-ServerRescueTestJSON-2068892925 tempest-ServerRescueTestJSON-2068892925-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 684.597043] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-f7bee0b7-3179-433d-8ad1-fa7369dca9b3 tempest-ServerRescueTestJSON-2068892925 tempest-ServerRescueTestJSON-2068892925-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 684.597043] nova-conductor[52243]: Traceback (most recent call last): [ 684.597043] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 684.597043] nova-conductor[52243]: return func(*args, **kwargs) [ 684.597043] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 684.597043] nova-conductor[52243]: selections = self._select_destinations( [ 684.597043] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 684.597043] nova-conductor[52243]: selections = self._schedule( [ 684.597043] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 684.597043] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 684.597043] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 684.597043] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 684.597043] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 684.597043] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 684.597599] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-f7bee0b7-3179-433d-8ad1-fa7369dca9b3 tempest-ServerRescueTestJSON-2068892925 tempest-ServerRescueTestJSON-2068892925-project-member] [instance: 0e7d3589-5c00-43b0-b3af-920958d1d6c1] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager [None req-0fdc151b-26fd-4a6a-8689-ca475e4c36ea tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 685.264059] nova-conductor[52244]: Traceback (most recent call last): [ 685.264059] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 685.264059] nova-conductor[52244]: return func(*args, **kwargs) [ 685.264059] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 685.264059] nova-conductor[52244]: selections = self._select_destinations( [ 685.264059] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 685.264059] nova-conductor[52244]: selections = self._schedule( [ 685.264059] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 685.264059] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 685.264059] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 685.264059] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 685.264059] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager [ 685.264059] nova-conductor[52244]: ERROR nova.conductor.manager [ 685.277501] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-0fdc151b-26fd-4a6a-8689-ca475e4c36ea tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 685.279893] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-0fdc151b-26fd-4a6a-8689-ca475e4c36ea tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 685.279893] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-0fdc151b-26fd-4a6a-8689-ca475e4c36ea tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 685.332808] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-0fdc151b-26fd-4a6a-8689-ca475e4c36ea tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] [instance: cb48da9d-6aa9-43ba-96ae-6328f505b007] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 685.336442] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-0fdc151b-26fd-4a6a-8689-ca475e4c36ea tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 685.336442] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-0fdc151b-26fd-4a6a-8689-ca475e4c36ea tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 685.336442] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-0fdc151b-26fd-4a6a-8689-ca475e4c36ea tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 685.348689] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-0fdc151b-26fd-4a6a-8689-ca475e4c36ea tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 685.348689] nova-conductor[52244]: Traceback (most recent call last): [ 685.348689] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 685.348689] nova-conductor[52244]: return func(*args, **kwargs) [ 685.348689] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 685.348689] nova-conductor[52244]: selections = self._select_destinations( [ 685.348689] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 685.348689] nova-conductor[52244]: selections = self._schedule( [ 685.348689] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 685.348689] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 685.348689] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 685.348689] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 685.348689] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 685.348689] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 685.349775] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-0fdc151b-26fd-4a6a-8689-ca475e4c36ea tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] [instance: cb48da9d-6aa9-43ba-96ae-6328f505b007] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager [None req-b3f89d5c-8a22-454f-aa67-becbd5414abf tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 686.529170] nova-conductor[52243]: Traceback (most recent call last): [ 686.529170] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 686.529170] nova-conductor[52243]: return func(*args, **kwargs) [ 686.529170] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 686.529170] nova-conductor[52243]: selections = self._select_destinations( [ 686.529170] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 686.529170] nova-conductor[52243]: selections = self._schedule( [ 686.529170] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 686.529170] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 686.529170] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 686.529170] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 686.529170] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager [ 686.529170] nova-conductor[52243]: ERROR nova.conductor.manager [ 686.536347] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b3f89d5c-8a22-454f-aa67-becbd5414abf tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 686.537468] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b3f89d5c-8a22-454f-aa67-becbd5414abf tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 686.540548] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b3f89d5c-8a22-454f-aa67-becbd5414abf tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 686.597728] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-b3f89d5c-8a22-454f-aa67-becbd5414abf tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] [instance: 1afef74c-34fd-4193-8533-9c2194f15b48] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 686.599796] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b3f89d5c-8a22-454f-aa67-becbd5414abf tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 686.600532] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b3f89d5c-8a22-454f-aa67-becbd5414abf tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 686.600532] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b3f89d5c-8a22-454f-aa67-becbd5414abf tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 686.604570] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-b3f89d5c-8a22-454f-aa67-becbd5414abf tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 686.604570] nova-conductor[52243]: Traceback (most recent call last): [ 686.604570] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 686.604570] nova-conductor[52243]: return func(*args, **kwargs) [ 686.604570] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 686.604570] nova-conductor[52243]: selections = self._select_destinations( [ 686.604570] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 686.604570] nova-conductor[52243]: selections = self._schedule( [ 686.604570] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 686.604570] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 686.604570] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 686.604570] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 686.604570] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 686.604570] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 686.605195] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-b3f89d5c-8a22-454f-aa67-becbd5414abf tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] [instance: 1afef74c-34fd-4193-8533-9c2194f15b48] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager [None req-f1574f70-906f-411c-b98c-916efcc1adb6 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 688.053675] nova-conductor[52244]: Traceback (most recent call last): [ 688.053675] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 688.053675] nova-conductor[52244]: return func(*args, **kwargs) [ 688.053675] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 688.053675] nova-conductor[52244]: selections = self._select_destinations( [ 688.053675] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 688.053675] nova-conductor[52244]: selections = self._schedule( [ 688.053675] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 688.053675] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 688.053675] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 688.053675] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 688.053675] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager [ 688.053675] nova-conductor[52244]: ERROR nova.conductor.manager [ 688.059402] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-f1574f70-906f-411c-b98c-916efcc1adb6 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 688.062884] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-f1574f70-906f-411c-b98c-916efcc1adb6 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 688.062884] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-f1574f70-906f-411c-b98c-916efcc1adb6 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 688.120561] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-f1574f70-906f-411c-b98c-916efcc1adb6 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] [instance: a55dc971-c3d8-4c44-856b-71b14cc66b82] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 688.121321] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-f1574f70-906f-411c-b98c-916efcc1adb6 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 688.121524] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-f1574f70-906f-411c-b98c-916efcc1adb6 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 688.121687] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-f1574f70-906f-411c-b98c-916efcc1adb6 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 688.126503] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-f1574f70-906f-411c-b98c-916efcc1adb6 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 688.126503] nova-conductor[52244]: Traceback (most recent call last): [ 688.126503] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 688.126503] nova-conductor[52244]: return func(*args, **kwargs) [ 688.126503] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 688.126503] nova-conductor[52244]: selections = self._select_destinations( [ 688.126503] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 688.126503] nova-conductor[52244]: selections = self._schedule( [ 688.126503] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 688.126503] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 688.126503] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 688.126503] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 688.126503] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 688.126503] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 688.127096] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-f1574f70-906f-411c-b98c-916efcc1adb6 tempest-ServersAdminTestJSON-675913171 tempest-ServersAdminTestJSON-675913171-project-member] [instance: a55dc971-c3d8-4c44-856b-71b14cc66b82] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager [None req-cf187ccc-88bb-4806-9fe0-8cc57e908c6c tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 688.717414] nova-conductor[52243]: Traceback (most recent call last): [ 688.717414] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 688.717414] nova-conductor[52243]: return func(*args, **kwargs) [ 688.717414] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 688.717414] nova-conductor[52243]: selections = self._select_destinations( [ 688.717414] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 688.717414] nova-conductor[52243]: selections = self._schedule( [ 688.717414] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 688.717414] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 688.717414] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 688.717414] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 688.717414] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager [ 688.717414] nova-conductor[52243]: ERROR nova.conductor.manager [ 688.725041] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-cf187ccc-88bb-4806-9fe0-8cc57e908c6c tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 688.725297] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-cf187ccc-88bb-4806-9fe0-8cc57e908c6c tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 688.727864] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-cf187ccc-88bb-4806-9fe0-8cc57e908c6c tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 688.786978] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-cf187ccc-88bb-4806-9fe0-8cc57e908c6c tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] [instance: c4362ac3-43c8-483b-8cf7-890fe3360eda] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 688.788057] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-cf187ccc-88bb-4806-9fe0-8cc57e908c6c tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 688.788057] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-cf187ccc-88bb-4806-9fe0-8cc57e908c6c tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 688.788168] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-cf187ccc-88bb-4806-9fe0-8cc57e908c6c tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 688.794424] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-cf187ccc-88bb-4806-9fe0-8cc57e908c6c tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 688.794424] nova-conductor[52243]: Traceback (most recent call last): [ 688.794424] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 688.794424] nova-conductor[52243]: return func(*args, **kwargs) [ 688.794424] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 688.794424] nova-conductor[52243]: selections = self._select_destinations( [ 688.794424] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 688.794424] nova-conductor[52243]: selections = self._schedule( [ 688.794424] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 688.794424] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 688.794424] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 688.794424] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 688.794424] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 688.794424] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 688.795235] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-cf187ccc-88bb-4806-9fe0-8cc57e908c6c tempest-ServersTestMultiNic-532910345 tempest-ServersTestMultiNic-532910345-project-member] [instance: c4362ac3-43c8-483b-8cf7-890fe3360eda] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager [None req-13e1c047-3504-4a17-83a8-07d922d63437 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 692.077114] nova-conductor[52244]: Traceback (most recent call last): [ 692.077114] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 692.077114] nova-conductor[52244]: return func(*args, **kwargs) [ 692.077114] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 692.077114] nova-conductor[52244]: selections = self._select_destinations( [ 692.077114] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 692.077114] nova-conductor[52244]: selections = self._schedule( [ 692.077114] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 692.077114] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 692.077114] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 692.077114] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 692.077114] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager [ 692.077114] nova-conductor[52244]: ERROR nova.conductor.manager [ 692.085124] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-13e1c047-3504-4a17-83a8-07d922d63437 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 692.085407] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-13e1c047-3504-4a17-83a8-07d922d63437 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 692.085641] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-13e1c047-3504-4a17-83a8-07d922d63437 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 692.138664] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-13e1c047-3504-4a17-83a8-07d922d63437 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] [instance: 289c148e-386a-4104-a536-05d7e552d319] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 692.139420] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-13e1c047-3504-4a17-83a8-07d922d63437 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 692.139625] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-13e1c047-3504-4a17-83a8-07d922d63437 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 692.139809] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-13e1c047-3504-4a17-83a8-07d922d63437 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 692.143394] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-13e1c047-3504-4a17-83a8-07d922d63437 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 692.143394] nova-conductor[52244]: Traceback (most recent call last): [ 692.143394] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 692.143394] nova-conductor[52244]: return func(*args, **kwargs) [ 692.143394] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 692.143394] nova-conductor[52244]: selections = self._select_destinations( [ 692.143394] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 692.143394] nova-conductor[52244]: selections = self._schedule( [ 692.143394] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 692.143394] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 692.143394] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 692.143394] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 692.143394] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 692.143394] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 692.143923] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-13e1c047-3504-4a17-83a8-07d922d63437 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] [instance: 289c148e-386a-4104-a536-05d7e552d319] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager [None req-ffc5fdd7-04d1-4627-92b5-fb76d599dd09 tempest-ImagesOneServerTestJSON-2065743937 tempest-ImagesOneServerTestJSON-2065743937-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 692.521164] nova-conductor[52243]: Traceback (most recent call last): [ 692.521164] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 692.521164] nova-conductor[52243]: return func(*args, **kwargs) [ 692.521164] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 692.521164] nova-conductor[52243]: selections = self._select_destinations( [ 692.521164] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 692.521164] nova-conductor[52243]: selections = self._schedule( [ 692.521164] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 692.521164] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 692.521164] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 692.521164] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 692.521164] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager [ 692.521164] nova-conductor[52243]: ERROR nova.conductor.manager [ 692.535147] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ffc5fdd7-04d1-4627-92b5-fb76d599dd09 tempest-ImagesOneServerTestJSON-2065743937 tempest-ImagesOneServerTestJSON-2065743937-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 692.535416] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ffc5fdd7-04d1-4627-92b5-fb76d599dd09 tempest-ImagesOneServerTestJSON-2065743937 tempest-ImagesOneServerTestJSON-2065743937-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 692.535592] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ffc5fdd7-04d1-4627-92b5-fb76d599dd09 tempest-ImagesOneServerTestJSON-2065743937 tempest-ImagesOneServerTestJSON-2065743937-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 692.601964] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-ffc5fdd7-04d1-4627-92b5-fb76d599dd09 tempest-ImagesOneServerTestJSON-2065743937 tempest-ImagesOneServerTestJSON-2065743937-project-member] [instance: d6cfab8e-8d89-40dd-90c8-ec837c4ad5be] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 692.602794] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ffc5fdd7-04d1-4627-92b5-fb76d599dd09 tempest-ImagesOneServerTestJSON-2065743937 tempest-ImagesOneServerTestJSON-2065743937-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 692.603035] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ffc5fdd7-04d1-4627-92b5-fb76d599dd09 tempest-ImagesOneServerTestJSON-2065743937 tempest-ImagesOneServerTestJSON-2065743937-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 692.603229] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-ffc5fdd7-04d1-4627-92b5-fb76d599dd09 tempest-ImagesOneServerTestJSON-2065743937 tempest-ImagesOneServerTestJSON-2065743937-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 692.606699] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-ffc5fdd7-04d1-4627-92b5-fb76d599dd09 tempest-ImagesOneServerTestJSON-2065743937 tempest-ImagesOneServerTestJSON-2065743937-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 692.606699] nova-conductor[52243]: Traceback (most recent call last): [ 692.606699] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 692.606699] nova-conductor[52243]: return func(*args, **kwargs) [ 692.606699] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 692.606699] nova-conductor[52243]: selections = self._select_destinations( [ 692.606699] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 692.606699] nova-conductor[52243]: selections = self._schedule( [ 692.606699] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 692.606699] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 692.606699] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 692.606699] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 692.606699] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 692.606699] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 692.607318] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-ffc5fdd7-04d1-4627-92b5-fb76d599dd09 tempest-ImagesOneServerTestJSON-2065743937 tempest-ImagesOneServerTestJSON-2065743937-project-member] [instance: d6cfab8e-8d89-40dd-90c8-ec837c4ad5be] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager [None req-023f987b-6bd1-44df-8a67-d93010099df1 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 695.688926] nova-conductor[52244]: Traceback (most recent call last): [ 695.688926] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 695.688926] nova-conductor[52244]: return func(*args, **kwargs) [ 695.688926] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 695.688926] nova-conductor[52244]: selections = self._select_destinations( [ 695.688926] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 695.688926] nova-conductor[52244]: selections = self._schedule( [ 695.688926] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 695.688926] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 695.688926] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 695.688926] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 695.688926] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager [ 695.688926] nova-conductor[52244]: ERROR nova.conductor.manager [ 695.699450] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-023f987b-6bd1-44df-8a67-d93010099df1 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 695.699885] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-023f987b-6bd1-44df-8a67-d93010099df1 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 695.700223] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-023f987b-6bd1-44df-8a67-d93010099df1 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 695.750655] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-023f987b-6bd1-44df-8a67-d93010099df1 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] [instance: c2a06ce5-763d-4d34-88fd-2b2e9d4b067f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 695.751479] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-023f987b-6bd1-44df-8a67-d93010099df1 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 695.752063] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-023f987b-6bd1-44df-8a67-d93010099df1 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 695.752300] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-023f987b-6bd1-44df-8a67-d93010099df1 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 695.755485] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-023f987b-6bd1-44df-8a67-d93010099df1 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 695.755485] nova-conductor[52244]: Traceback (most recent call last): [ 695.755485] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 695.755485] nova-conductor[52244]: return func(*args, **kwargs) [ 695.755485] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 695.755485] nova-conductor[52244]: selections = self._select_destinations( [ 695.755485] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 695.755485] nova-conductor[52244]: selections = self._schedule( [ 695.755485] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 695.755485] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 695.755485] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 695.755485] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 695.755485] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 695.755485] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 695.756069] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-023f987b-6bd1-44df-8a67-d93010099df1 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] [instance: c2a06ce5-763d-4d34-88fd-2b2e9d4b067f] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager [None req-804c0b3c-db16-4079-b8e0-fa0f9b1ff7ec tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 698.297307] nova-conductor[52244]: Traceback (most recent call last): [ 698.297307] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 698.297307] nova-conductor[52244]: return func(*args, **kwargs) [ 698.297307] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 698.297307] nova-conductor[52244]: selections = self._select_destinations( [ 698.297307] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 698.297307] nova-conductor[52244]: selections = self._schedule( [ 698.297307] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 698.297307] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 698.297307] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 698.297307] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 698.297307] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager [ 698.297307] nova-conductor[52244]: ERROR nova.conductor.manager [ 698.304762] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-804c0b3c-db16-4079-b8e0-fa0f9b1ff7ec tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 698.305467] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-804c0b3c-db16-4079-b8e0-fa0f9b1ff7ec tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 698.305467] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-804c0b3c-db16-4079-b8e0-fa0f9b1ff7ec tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 698.359178] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-804c0b3c-db16-4079-b8e0-fa0f9b1ff7ec tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: a0cadc05-539b-4f36-9b19-8aec4c5f92b6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 698.360280] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-804c0b3c-db16-4079-b8e0-fa0f9b1ff7ec tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 698.361421] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-804c0b3c-db16-4079-b8e0-fa0f9b1ff7ec tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 698.361421] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-804c0b3c-db16-4079-b8e0-fa0f9b1ff7ec tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 698.365568] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-804c0b3c-db16-4079-b8e0-fa0f9b1ff7ec tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 698.365568] nova-conductor[52244]: Traceback (most recent call last): [ 698.365568] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 698.365568] nova-conductor[52244]: return func(*args, **kwargs) [ 698.365568] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 698.365568] nova-conductor[52244]: selections = self._select_destinations( [ 698.365568] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 698.365568] nova-conductor[52244]: selections = self._schedule( [ 698.365568] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 698.365568] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 698.365568] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 698.365568] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 698.365568] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 698.365568] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 698.365568] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-804c0b3c-db16-4079-b8e0-fa0f9b1ff7ec tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: a0cadc05-539b-4f36-9b19-8aec4c5f92b6] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager [None req-7f5f8d90-28ec-49fb-82ea-a7851964f3bc tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 700.271835] nova-conductor[52243]: Traceback (most recent call last): [ 700.271835] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 700.271835] nova-conductor[52243]: return func(*args, **kwargs) [ 700.271835] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 700.271835] nova-conductor[52243]: selections = self._select_destinations( [ 700.271835] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 700.271835] nova-conductor[52243]: selections = self._schedule( [ 700.271835] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 700.271835] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 700.271835] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 700.271835] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 700.271835] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager [ 700.271835] nova-conductor[52243]: ERROR nova.conductor.manager [ 700.280622] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-7f5f8d90-28ec-49fb-82ea-a7851964f3bc tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 700.280622] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-7f5f8d90-28ec-49fb-82ea-a7851964f3bc tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 700.280622] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-7f5f8d90-28ec-49fb-82ea-a7851964f3bc tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 700.343144] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-7f5f8d90-28ec-49fb-82ea-a7851964f3bc tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] [instance: 081de977-9854-4e53-b7e3-425cedbe2f75] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 700.343975] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-7f5f8d90-28ec-49fb-82ea-a7851964f3bc tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 700.344368] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-7f5f8d90-28ec-49fb-82ea-a7851964f3bc tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 700.345098] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-7f5f8d90-28ec-49fb-82ea-a7851964f3bc tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 700.352743] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-7f5f8d90-28ec-49fb-82ea-a7851964f3bc tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 700.352743] nova-conductor[52243]: Traceback (most recent call last): [ 700.352743] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 700.352743] nova-conductor[52243]: return func(*args, **kwargs) [ 700.352743] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 700.352743] nova-conductor[52243]: selections = self._select_destinations( [ 700.352743] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 700.352743] nova-conductor[52243]: selections = self._schedule( [ 700.352743] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 700.352743] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 700.352743] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 700.352743] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 700.352743] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 700.352743] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 700.356268] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-7f5f8d90-28ec-49fb-82ea-a7851964f3bc tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] [instance: 081de977-9854-4e53-b7e3-425cedbe2f75] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager [None req-72a57f64-bdf8-4b8c-b4d5-eb5aaf5b2c99 tempest-ServersNegativeTestMultiTenantJSON-1661334709 tempest-ServersNegativeTestMultiTenantJSON-1661334709-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 700.808305] nova-conductor[52244]: Traceback (most recent call last): [ 700.808305] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 700.808305] nova-conductor[52244]: return func(*args, **kwargs) [ 700.808305] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 700.808305] nova-conductor[52244]: selections = self._select_destinations( [ 700.808305] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 700.808305] nova-conductor[52244]: selections = self._schedule( [ 700.808305] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 700.808305] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 700.808305] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 700.808305] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 700.808305] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager [ 700.808305] nova-conductor[52244]: ERROR nova.conductor.manager [ 700.816869] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-72a57f64-bdf8-4b8c-b4d5-eb5aaf5b2c99 tempest-ServersNegativeTestMultiTenantJSON-1661334709 tempest-ServersNegativeTestMultiTenantJSON-1661334709-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 700.816869] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-72a57f64-bdf8-4b8c-b4d5-eb5aaf5b2c99 tempest-ServersNegativeTestMultiTenantJSON-1661334709 tempest-ServersNegativeTestMultiTenantJSON-1661334709-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 700.816869] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-72a57f64-bdf8-4b8c-b4d5-eb5aaf5b2c99 tempest-ServersNegativeTestMultiTenantJSON-1661334709 tempest-ServersNegativeTestMultiTenantJSON-1661334709-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 700.862587] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-72a57f64-bdf8-4b8c-b4d5-eb5aaf5b2c99 tempest-ServersNegativeTestMultiTenantJSON-1661334709 tempest-ServersNegativeTestMultiTenantJSON-1661334709-project-member] [instance: 79046690-e296-49cf-86f7-aefb1b9d23ff] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 700.863345] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-72a57f64-bdf8-4b8c-b4d5-eb5aaf5b2c99 tempest-ServersNegativeTestMultiTenantJSON-1661334709 tempest-ServersNegativeTestMultiTenantJSON-1661334709-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 700.863563] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-72a57f64-bdf8-4b8c-b4d5-eb5aaf5b2c99 tempest-ServersNegativeTestMultiTenantJSON-1661334709 tempest-ServersNegativeTestMultiTenantJSON-1661334709-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 700.863730] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-72a57f64-bdf8-4b8c-b4d5-eb5aaf5b2c99 tempest-ServersNegativeTestMultiTenantJSON-1661334709 tempest-ServersNegativeTestMultiTenantJSON-1661334709-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 700.870140] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-72a57f64-bdf8-4b8c-b4d5-eb5aaf5b2c99 tempest-ServersNegativeTestMultiTenantJSON-1661334709 tempest-ServersNegativeTestMultiTenantJSON-1661334709-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 700.870140] nova-conductor[52244]: Traceback (most recent call last): [ 700.870140] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 700.870140] nova-conductor[52244]: return func(*args, **kwargs) [ 700.870140] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 700.870140] nova-conductor[52244]: selections = self._select_destinations( [ 700.870140] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 700.870140] nova-conductor[52244]: selections = self._schedule( [ 700.870140] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 700.870140] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 700.870140] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 700.870140] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 700.870140] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 700.870140] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 700.870140] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-72a57f64-bdf8-4b8c-b4d5-eb5aaf5b2c99 tempest-ServersNegativeTestMultiTenantJSON-1661334709 tempest-ServersNegativeTestMultiTenantJSON-1661334709-project-member] [instance: 79046690-e296-49cf-86f7-aefb1b9d23ff] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager [None req-a65b451a-838b-47d9-8b0b-15bd68adfefe tempest-ServerActionsTestOtherB-1433247059 tempest-ServerActionsTestOtherB-1433247059-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 703.201308] nova-conductor[52243]: Traceback (most recent call last): [ 703.201308] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 703.201308] nova-conductor[52243]: return func(*args, **kwargs) [ 703.201308] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 703.201308] nova-conductor[52243]: selections = self._select_destinations( [ 703.201308] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 703.201308] nova-conductor[52243]: selections = self._schedule( [ 703.201308] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 703.201308] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 703.201308] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 703.201308] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 703.201308] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager [ 703.201308] nova-conductor[52243]: ERROR nova.conductor.manager [ 703.210722] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a65b451a-838b-47d9-8b0b-15bd68adfefe tempest-ServerActionsTestOtherB-1433247059 tempest-ServerActionsTestOtherB-1433247059-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 703.210818] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a65b451a-838b-47d9-8b0b-15bd68adfefe tempest-ServerActionsTestOtherB-1433247059 tempest-ServerActionsTestOtherB-1433247059-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 703.210933] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a65b451a-838b-47d9-8b0b-15bd68adfefe tempest-ServerActionsTestOtherB-1433247059 tempest-ServerActionsTestOtherB-1433247059-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 703.262098] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-a65b451a-838b-47d9-8b0b-15bd68adfefe tempest-ServerActionsTestOtherB-1433247059 tempest-ServerActionsTestOtherB-1433247059-project-member] [instance: c86a0f45-516c-47c3-a012-f56839e05e91] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 703.262098] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a65b451a-838b-47d9-8b0b-15bd68adfefe tempest-ServerActionsTestOtherB-1433247059 tempest-ServerActionsTestOtherB-1433247059-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 703.262467] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a65b451a-838b-47d9-8b0b-15bd68adfefe tempest-ServerActionsTestOtherB-1433247059 tempest-ServerActionsTestOtherB-1433247059-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 703.263095] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a65b451a-838b-47d9-8b0b-15bd68adfefe tempest-ServerActionsTestOtherB-1433247059 tempest-ServerActionsTestOtherB-1433247059-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 703.273301] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-a65b451a-838b-47d9-8b0b-15bd68adfefe tempest-ServerActionsTestOtherB-1433247059 tempest-ServerActionsTestOtherB-1433247059-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 703.273301] nova-conductor[52243]: Traceback (most recent call last): [ 703.273301] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 703.273301] nova-conductor[52243]: return func(*args, **kwargs) [ 703.273301] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 703.273301] nova-conductor[52243]: selections = self._select_destinations( [ 703.273301] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 703.273301] nova-conductor[52243]: selections = self._schedule( [ 703.273301] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 703.273301] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 703.273301] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 703.273301] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 703.273301] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 703.273301] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 703.274952] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-a65b451a-838b-47d9-8b0b-15bd68adfefe tempest-ServerActionsTestOtherB-1433247059 tempest-ServerActionsTestOtherB-1433247059-project-member] [instance: c86a0f45-516c-47c3-a012-f56839e05e91] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 710.339945] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 710.352862] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.353227] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.353338] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 710.425177] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.425584] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.425851] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 710.426394] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.426701] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.426953] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 710.437394] nova-conductor[52243]: DEBUG nova.quota [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Getting quotas for project 834aaa625fd84a18b32ddef466b431a4. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 710.440288] nova-conductor[52243]: DEBUG nova.quota [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Getting quotas for user 7f4810d7f5164775b9ef4b51ea0883b6 and project 834aaa625fd84a18b32ddef466b431a4. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 710.447235] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] [instance: ea968312-62ea-4f55-87e9-f91823fc14c2] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 710.447844] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.448609] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.448609] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 710.458039] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] [instance: ea968312-62ea-4f55-87e9-f91823fc14c2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 710.458748] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.459451] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.459451] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 710.481647] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.481878] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.483360] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 711.684769] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Took 0.18 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 711.719437] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 711.719437] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 711.719437] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 711.767609] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 711.767903] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 711.768218] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 711.768482] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 711.768666] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 711.768925] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 711.788020] nova-conductor[52243]: DEBUG nova.quota [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Getting quotas for project e235144751dc4f10be76b379d3e6e53d. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 711.792869] nova-conductor[52243]: DEBUG nova.quota [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Getting quotas for user 63ed5c849ba14360bfc1356ce027a979 and project e235144751dc4f10be76b379d3e6e53d. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 711.801829] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] [instance: 9759f284-26e2-466e-9504-ffb63a359f27] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 711.802597] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 711.802827] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 711.803011] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 711.809016] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] [instance: 9759f284-26e2-466e-9504-ffb63a359f27] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 711.809016] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 711.809016] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 711.809016] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 711.828119] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 711.828395] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 711.828530] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 712.339925] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Took 0.21 seconds to select destinations for 1 instance(s). {{(pid=52244) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 712.366187] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 712.366472] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 712.366768] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 712.400899] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 712.400899] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 712.401070] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 712.401442] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 712.401648] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 712.401812] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 712.411228] nova-conductor[52244]: DEBUG nova.quota [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Getting quotas for project 580e685106a749aca0f9769c5c269798. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 712.413565] nova-conductor[52244]: DEBUG nova.quota [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Getting quotas for user e9d0f0c1831a4246b24095e2592c8cfc and project 580e685106a749aca0f9769c5c269798. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 712.423128] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] [instance: fdd34513-15af-4294-8a8a-e3b095188eda] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52244) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 712.423128] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 712.423128] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 712.423264] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 712.426426] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] [instance: fdd34513-15af-4294-8a8a-e3b095188eda] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 712.427344] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 712.427344] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 712.427521] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 712.444193] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 712.444872] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 712.445132] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 713.503268] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 713.519103] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 713.519203] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 713.519372] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 713.566429] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 713.566667] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 713.566845] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 713.567229] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 713.567415] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 713.567578] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 713.582000] nova-conductor[52243]: DEBUG nova.quota [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Getting quotas for project 20659337b9974b26a9f4123181bd14ce. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 713.584678] nova-conductor[52243]: DEBUG nova.quota [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Getting quotas for user 4f3796dbcf6c46a4be35270ec92c09da and project 20659337b9974b26a9f4123181bd14ce. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 713.590381] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] [instance: 62ade33c-5283-432d-872c-cc162254317d] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 713.591405] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 713.591405] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 713.591405] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 713.594277] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] [instance: 62ade33c-5283-432d-872c-cc162254317d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 713.594926] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 713.595139] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 713.595302] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 713.617579] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 713.618324] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 713.618324] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 714.805012] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Took 0.13 seconds to select destinations for 1 instance(s). {{(pid=52244) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 714.816897] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.817358] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.817358] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 714.854047] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.854047] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.854047] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 714.854047] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.854047] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.854047] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 714.863280] nova-conductor[52244]: DEBUG nova.quota [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Getting quotas for project 291b1c241186446bb53562f866315ad9. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 714.865806] nova-conductor[52244]: DEBUG nova.quota [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Getting quotas for user 4dab02812c6c4fed99d165cd787c842c and project 291b1c241186446bb53562f866315ad9. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 714.871890] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] [instance: ce3bd633-4538-428d-9258-9222c3c72edd] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52244) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 714.872394] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.874162] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.874162] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 714.876876] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] [instance: ce3bd633-4538-428d-9258-9222c3c72edd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 714.877879] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.878205] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.878546] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 714.893250] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.893489] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.893678] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 716.139958] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Took 0.15 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 716.154505] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 716.154787] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 716.155029] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 716.161476] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Took 0.15 seconds to select destinations for 1 instance(s). {{(pid=52244) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 716.183849] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 716.183849] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 716.183849] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 716.192395] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 716.192710] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 716.192928] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 716.193321] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 716.193557] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 716.193769] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 716.202801] nova-conductor[52243]: DEBUG nova.quota [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Getting quotas for project 8689e7ba4d544dfcbbdf7c864cb3f823. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 716.205472] nova-conductor[52243]: DEBUG nova.quota [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Getting quotas for user 8090dd0e116e4ac89aeb07e25bc22927 and project 8689e7ba4d544dfcbbdf7c864cb3f823. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 716.208703] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 716.209063] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 716.209390] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 716.211029] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 716.211323] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 716.211620] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 716.216018] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: f7fc4465-02a5-4715-b50b-04172f097350] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 716.216018] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 716.216018] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 716.216018] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 716.217665] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: f7fc4465-02a5-4715-b50b-04172f097350] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 716.218587] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 716.218859] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 716.219046] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 716.221672] nova-conductor[52244]: DEBUG nova.quota [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Getting quotas for project dd2add382fb34e309cc9b0acd9403ef6. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 716.226743] nova-conductor[52244]: DEBUG nova.quota [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Getting quotas for user 7c2b5f71e3034e5f90220c5ebf1bb6d9 and project dd2add382fb34e309cc9b0acd9403ef6. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 716.232337] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 716.232510] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 716.232705] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 716.233843] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: 588ad4c7-67fd-4d4f-ae0f-bb1ded40ef06] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52244) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 716.234469] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 716.234996] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 716.235289] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 716.241133] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: 588ad4c7-67fd-4d4f-ae0f-bb1ded40ef06] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 716.241133] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 716.241133] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 716.241133] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 716.255583] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 716.255805] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 716.255976] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 719.803651] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 719.823324] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 719.824121] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 719.824121] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 719.864105] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 719.867020] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 719.867020] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 719.867020] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 719.867020] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 719.867020] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 719.876292] nova-conductor[52243]: DEBUG nova.quota [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Getting quotas for project b04115b4b0ed49aea28a1bee3ee07a27. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 719.881031] nova-conductor[52243]: DEBUG nova.quota [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Getting quotas for user 18cab6bcb60c42d48d6cb367a7d2406c and project b04115b4b0ed49aea28a1bee3ee07a27. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 719.891318] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] [instance: 1f7f6276-cfe5-4427-90b6-893e7ad6cffe] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 719.891318] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 719.891318] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 719.891318] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 719.893876] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] [instance: 1f7f6276-cfe5-4427-90b6-893e7ad6cffe] block_device_mapping [BlockDeviceMapping(attachment_id=bc4a410a-054e-4f2e-a8f1-0dcd32213ee5,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='a29828fb-9b19-4403-8dcf-5f30e3e7298c',volume_size=1,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 719.894906] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 719.895248] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 719.895531] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 719.910393] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 719.910875] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 719.911167] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 721.109083] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Took 0.15 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 721.123276] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 721.123503] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 721.123675] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 721.167303] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 721.168953] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 721.168953] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 721.168953] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 721.168953] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 721.168953] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 721.177901] nova-conductor[52243]: DEBUG nova.quota [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Getting quotas for project cd4ace35259947bfa4d07c4c3f42d55b. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 721.180464] nova-conductor[52243]: DEBUG nova.quota [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Getting quotas for user 307ded03aa7142f1bb4b8134a4dbb3f8 and project cd4ace35259947bfa4d07c4c3f42d55b. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 721.186582] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] [instance: ad58bbc3-1ec8-4567-ba07-c8161bcc8380] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 721.187128] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 721.187342] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 721.187556] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 721.191010] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] [instance: ad58bbc3-1ec8-4567-ba07-c8161bcc8380] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 721.191010] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 721.193482] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 721.193482] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 721.205471] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 721.205694] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 721.205893] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 722.331479] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 722.346889] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 722.347209] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 722.347431] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 722.380224] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 722.380454] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 722.380625] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 722.380978] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 722.381175] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 722.381336] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 722.389465] nova-conductor[52243]: DEBUG nova.quota [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Getting quotas for project f858ee4b23fb49c399f103c4a8bcdebc. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 722.391897] nova-conductor[52243]: DEBUG nova.quota [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Getting quotas for user b8d25c6dcda1421b82c920c9580bf020 and project f858ee4b23fb49c399f103c4a8bcdebc. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 722.398316] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] [instance: 7b7c9dcb-3ddd-419e-a6fa-6b37b01cc998] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 722.398798] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 722.399158] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 722.399254] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 722.405460] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] [instance: 7b7c9dcb-3ddd-419e-a6fa-6b37b01cc998] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 722.405881] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 722.406110] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 722.406280] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 722.421446] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 722.421655] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 722.421814] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 722.827187] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] [instance: fdd34513-15af-4294-8a8a-e3b095188eda] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 711f0237-f381-4494-9f23-9a9c2e51d498, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance fdd34513-15af-4294-8a8a-e3b095188eda was re-scheduled: Binding failed for port 711f0237-f381-4494-9f23-9a9c2e51d498, please check neutron logs for more information.\n'] [ 722.827889] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 722.828194] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fdd34513-15af-4294-8a8a-e3b095188eda.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fdd34513-15af-4294-8a8a-e3b095188eda. [ 722.828564] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] [instance: fdd34513-15af-4294-8a8a-e3b095188eda] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fdd34513-15af-4294-8a8a-e3b095188eda. [ 722.854712] nova-conductor[52244]: DEBUG nova.network.neutron [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] [instance: fdd34513-15af-4294-8a8a-e3b095188eda] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 722.967946] nova-conductor[52244]: DEBUG nova.network.neutron [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] [instance: fdd34513-15af-4294-8a8a-e3b095188eda] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 722.972183] nova-conductor[52244]: DEBUG nova.network.neutron [None req-fb650bf2-8157-43bb-9fcc-30774c4f895c tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] [instance: fdd34513-15af-4294-8a8a-e3b095188eda] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 723.740519] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] [instance: 62ade33c-5283-432d-872c-cc162254317d] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 88dc529c-e6b7-4d6e-b7a7-17e32a9977fd, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 62ade33c-5283-432d-872c-cc162254317d was re-scheduled: Binding failed for port 88dc529c-e6b7-4d6e-b7a7-17e32a9977fd, please check neutron logs for more information.\n'] [ 723.740519] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 723.740519] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 62ade33c-5283-432d-872c-cc162254317d.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 62ade33c-5283-432d-872c-cc162254317d. [ 723.740937] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] [instance: 62ade33c-5283-432d-872c-cc162254317d] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 62ade33c-5283-432d-872c-cc162254317d. [ 723.760378] nova-conductor[52244]: DEBUG nova.network.neutron [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] [instance: 62ade33c-5283-432d-872c-cc162254317d] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 724.063076] nova-conductor[52244]: DEBUG nova.network.neutron [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] [instance: 62ade33c-5283-432d-872c-cc162254317d] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 724.066886] nova-conductor[52244]: DEBUG nova.network.neutron [None req-8d1248a1-05de-4a70-8c0b-9763d4c8c4e3 tempest-VolumesAdminNegativeTest-116106725 tempest-VolumesAdminNegativeTest-116106725-project-member] [instance: 62ade33c-5283-432d-872c-cc162254317d] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 724.556209] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] [instance: 9759f284-26e2-466e-9504-ffb63a359f27] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port f96d3967-9568-40ee-9bd9-a29f08464a46, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 9759f284-26e2-466e-9504-ffb63a359f27 was re-scheduled: Binding failed for port f96d3967-9568-40ee-9bd9-a29f08464a46, please check neutron logs for more information.\n'] [ 724.556209] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 724.556209] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9759f284-26e2-466e-9504-ffb63a359f27.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9759f284-26e2-466e-9504-ffb63a359f27. [ 724.556209] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] [instance: 9759f284-26e2-466e-9504-ffb63a359f27] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9759f284-26e2-466e-9504-ffb63a359f27. [ 724.587028] nova-conductor[52244]: DEBUG nova.network.neutron [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] [instance: 9759f284-26e2-466e-9504-ffb63a359f27] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 724.658861] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Took 0.15 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 724.670703] nova-conductor[52244]: DEBUG nova.network.neutron [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] [instance: 9759f284-26e2-466e-9504-ffb63a359f27] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 724.672523] nova-conductor[52244]: DEBUG nova.network.neutron [None req-04894b72-62f4-42ce-bcd9-93286e30827e tempest-ServersAdminNegativeTestJSON-370113334 tempest-ServersAdminNegativeTestJSON-370113334-project-member] [instance: 9759f284-26e2-466e-9504-ffb63a359f27] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 724.673749] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 724.673991] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 724.674203] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 724.716928] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 724.717187] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 724.717357] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 724.717716] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 724.717901] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 724.719269] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 724.731799] nova-conductor[52243]: DEBUG nova.quota [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Getting quotas for project 91232108c8944a3da00233e9c54c9749. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 724.734817] nova-conductor[52243]: DEBUG nova.quota [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Getting quotas for user 593629524d524ad9b515d92b36e7b1e8 and project 91232108c8944a3da00233e9c54c9749. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 724.740974] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: a411c5e7-5a49-463e-b270-800e35a31188] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 724.741500] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 724.741709] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 724.741875] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 724.745183] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: a411c5e7-5a49-463e-b270-800e35a31188] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 724.745887] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 724.746109] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 724.746294] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 724.760566] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 724.760566] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 724.760648] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 728.599374] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=52244) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 728.615839] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] [instance: 1f7f6276-cfe5-4427-90b6-893e7ad6cffe] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 5094a1db-1c30-434f-bb1a-00afbf387d0d, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 1f7f6276-cfe5-4427-90b6-893e7ad6cffe was re-scheduled: Binding failed for port 5094a1db-1c30-434f-bb1a-00afbf387d0d, please check neutron logs for more information.\n'] [ 728.616381] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 728.616623] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 1f7f6276-cfe5-4427-90b6-893e7ad6cffe.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 1f7f6276-cfe5-4427-90b6-893e7ad6cffe. [ 728.616867] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] [instance: 1f7f6276-cfe5-4427-90b6-893e7ad6cffe] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 1f7f6276-cfe5-4427-90b6-893e7ad6cffe. [ 728.623453] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 728.623693] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 728.623890] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 728.656899] nova-conductor[52244]: DEBUG nova.network.neutron [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] [instance: 1f7f6276-cfe5-4427-90b6-893e7ad6cffe] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 728.678942] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 728.680033] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 728.680033] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 728.680033] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 728.680033] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 728.680033] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 728.694486] nova-conductor[52244]: DEBUG nova.quota [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Getting quotas for project aa0980bc89de477bbd5ad4ad8f6ce2b4. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 728.697966] nova-conductor[52244]: DEBUG nova.quota [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Getting quotas for user 5efb0e88d47b4a648b96635f0901c069 and project aa0980bc89de477bbd5ad4ad8f6ce2b4. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 728.703797] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] [instance: 1670e7a3-656a-444d-85ed-292956498612] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52244) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 728.704268] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 728.704477] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 728.704645] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 728.707460] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] [instance: 1670e7a3-656a-444d-85ed-292956498612] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 728.708115] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 728.708324] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 728.708489] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 728.727092] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 728.727092] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 728.727092] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 728.877759] nova-conductor[52244]: DEBUG nova.network.neutron [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] [instance: 1f7f6276-cfe5-4427-90b6-893e7ad6cffe] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 728.882087] nova-conductor[52244]: DEBUG nova.network.neutron [None req-0098a2d1-2506-4fba-9164-111089f7368c tempest-ServersTestBootFromVolume-1432150623 tempest-ServersTestBootFromVolume-1432150623-project-member] [instance: 1f7f6276-cfe5-4427-90b6-893e7ad6cffe] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 729.398467] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] [instance: ce3bd633-4538-428d-9258-9222c3c72edd] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 808c0d62-376a-4bfe-a50c-92a1fa8874f0, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ce3bd633-4538-428d-9258-9222c3c72edd was re-scheduled: Binding failed for port 808c0d62-376a-4bfe-a50c-92a1fa8874f0, please check neutron logs for more information.\n'] [ 729.399046] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 729.399279] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ce3bd633-4538-428d-9258-9222c3c72edd.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ce3bd633-4538-428d-9258-9222c3c72edd. [ 729.399620] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] [instance: ce3bd633-4538-428d-9258-9222c3c72edd] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ce3bd633-4538-428d-9258-9222c3c72edd. [ 729.435970] nova-conductor[52244]: DEBUG nova.network.neutron [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] [instance: ce3bd633-4538-428d-9258-9222c3c72edd] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 729.500518] nova-conductor[52244]: DEBUG nova.network.neutron [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] [instance: ce3bd633-4538-428d-9258-9222c3c72edd] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 729.502886] nova-conductor[52244]: DEBUG nova.network.neutron [None req-a23dfe6a-4389-4ef3-ba9e-9ebf54fba058 tempest-InstanceActionsNegativeTestJSON-307740395 tempest-InstanceActionsNegativeTestJSON-307740395-project-member] [instance: ce3bd633-4538-428d-9258-9222c3c72edd] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 730.072060] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] [instance: 7b7c9dcb-3ddd-419e-a6fa-6b37b01cc998] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port dbbd873b-748b-4380-b79b-0889fea0b6d1, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 7b7c9dcb-3ddd-419e-a6fa-6b37b01cc998 was re-scheduled: Binding failed for port dbbd873b-748b-4380-b79b-0889fea0b6d1, please check neutron logs for more information.\n'] [ 730.072644] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 730.073110] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7b7c9dcb-3ddd-419e-a6fa-6b37b01cc998.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7b7c9dcb-3ddd-419e-a6fa-6b37b01cc998. [ 730.073229] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] [instance: 7b7c9dcb-3ddd-419e-a6fa-6b37b01cc998] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7b7c9dcb-3ddd-419e-a6fa-6b37b01cc998. [ 730.100022] nova-conductor[52244]: DEBUG nova.network.neutron [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] [instance: 7b7c9dcb-3ddd-419e-a6fa-6b37b01cc998] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 730.164625] nova-conductor[52244]: DEBUG nova.network.neutron [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] [instance: 7b7c9dcb-3ddd-419e-a6fa-6b37b01cc998] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 730.170263] nova-conductor[52244]: DEBUG nova.network.neutron [None req-333acd96-25f6-4d5e-b8a6-27bcdb473198 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] [instance: 7b7c9dcb-3ddd-419e-a6fa-6b37b01cc998] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 730.695095] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Took 0.15 seconds to select destinations for 1 instance(s). {{(pid=52244) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 730.711044] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 730.711044] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 730.711044] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 730.747309] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 730.747593] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 730.747767] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 730.748149] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 730.748337] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 730.748494] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 730.765865] nova-conductor[52244]: DEBUG nova.quota [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Getting quotas for project 319a39cba8574bc7a3c92d4527f59cad. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 730.768851] nova-conductor[52244]: DEBUG nova.quota [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Getting quotas for user 9c848bf8c29f415e96eee55939826dde and project 319a39cba8574bc7a3c92d4527f59cad. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 730.781402] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] [instance: ee7de02d-e1c1-4fe7-8df9-e9f82a39ef24] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52244) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 730.784019] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 730.784019] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 730.784019] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 730.786278] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] [instance: ee7de02d-e1c1-4fe7-8df9-e9f82a39ef24] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 730.786969] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 730.787177] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 730.787339] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 730.819710] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 730.819956] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 730.820148] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 731.687539] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: f7fc4465-02a5-4715-b50b-04172f097350] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 8eed8767-c291-4cbf-8803-07bd0caa822b, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance f7fc4465-02a5-4715-b50b-04172f097350 was re-scheduled: Binding failed for port 8eed8767-c291-4cbf-8803-07bd0caa822b, please check neutron logs for more information.\n'] [ 731.688473] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 731.688473] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f7fc4465-02a5-4715-b50b-04172f097350.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f7fc4465-02a5-4715-b50b-04172f097350. [ 731.689447] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: f7fc4465-02a5-4715-b50b-04172f097350] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f7fc4465-02a5-4715-b50b-04172f097350. [ 731.708181] nova-conductor[52244]: DEBUG nova.network.neutron [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: f7fc4465-02a5-4715-b50b-04172f097350] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 731.754437] nova-conductor[52244]: DEBUG nova.network.neutron [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: f7fc4465-02a5-4715-b50b-04172f097350] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 731.757553] nova-conductor[52244]: DEBUG nova.network.neutron [None req-bfee6349-6a74-41ed-8ad7-a6e50bc9516d tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: f7fc4465-02a5-4715-b50b-04172f097350] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 731.783720] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: 588ad4c7-67fd-4d4f-ae0f-bb1ded40ef06] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 66b70acf-8cb8-4462-b90d-f49fb9026d15, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 588ad4c7-67fd-4d4f-ae0f-bb1ded40ef06 was re-scheduled: Binding failed for port 66b70acf-8cb8-4462-b90d-f49fb9026d15, please check neutron logs for more information.\n'] [ 731.784385] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 731.784615] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 588ad4c7-67fd-4d4f-ae0f-bb1ded40ef06.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 588ad4c7-67fd-4d4f-ae0f-bb1ded40ef06. [ 731.784957] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: 588ad4c7-67fd-4d4f-ae0f-bb1ded40ef06] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 588ad4c7-67fd-4d4f-ae0f-bb1ded40ef06. [ 731.827058] nova-conductor[52243]: DEBUG nova.network.neutron [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: 588ad4c7-67fd-4d4f-ae0f-bb1ded40ef06] deallocate_for_instance() {{(pid=52243) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 731.883832] nova-conductor[52243]: DEBUG nova.network.neutron [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: 588ad4c7-67fd-4d4f-ae0f-bb1ded40ef06] Instance cache missing network info. {{(pid=52243) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 731.887667] nova-conductor[52243]: DEBUG nova.network.neutron [None req-2185c308-83d0-4687-8494-f4cfcd93bb63 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: 588ad4c7-67fd-4d4f-ae0f-bb1ded40ef06] Updating instance_info_cache with network_info: [] {{(pid=52243) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 731.984168] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Took 0.14 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 731.996448] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 731.996699] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 731.996873] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 732.034019] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 732.034019] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 732.034019] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 732.034019] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 732.034019] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 732.034019] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 732.043640] nova-conductor[52243]: DEBUG nova.quota [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Getting quotas for project cefaf2236f784faaa0f6d81a0ccda6f8. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 732.046379] nova-conductor[52243]: DEBUG nova.quota [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Getting quotas for user 0dcc88335cd04d18a843d54d7892e451 and project cefaf2236f784faaa0f6d81a0ccda6f8. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 732.052100] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] [instance: e90ee443-efe0-4f3e-999b-b9376e41fcb5] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 732.052547] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 732.052756] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 732.052921] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 732.061586] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] [instance: e90ee443-efe0-4f3e-999b-b9376e41fcb5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 732.061586] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 732.061586] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 732.061586] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 732.085287] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 732.085747] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 732.085747] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 734.381867] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] [instance: ad58bbc3-1ec8-4567-ba07-c8161bcc8380] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 346f3b4a-8fe0-46e9-8eb3-97e76ce72c7f, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ad58bbc3-1ec8-4567-ba07-c8161bcc8380 was re-scheduled: Binding failed for port 346f3b4a-8fe0-46e9-8eb3-97e76ce72c7f, please check neutron logs for more information.\n'] [ 734.382571] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 734.382843] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ad58bbc3-1ec8-4567-ba07-c8161bcc8380.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ad58bbc3-1ec8-4567-ba07-c8161bcc8380. [ 734.383209] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] [instance: ad58bbc3-1ec8-4567-ba07-c8161bcc8380] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ad58bbc3-1ec8-4567-ba07-c8161bcc8380. [ 734.423802] nova-conductor[52243]: DEBUG nova.network.neutron [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] [instance: ad58bbc3-1ec8-4567-ba07-c8161bcc8380] deallocate_for_instance() {{(pid=52243) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 734.558230] nova-conductor[52243]: DEBUG nova.network.neutron [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] [instance: ad58bbc3-1ec8-4567-ba07-c8161bcc8380] Instance cache missing network info. {{(pid=52243) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 734.569048] nova-conductor[52243]: DEBUG nova.network.neutron [None req-5dbb8e33-93eb-4bbc-9d22-46826868e029 tempest-AttachInterfacesTestJSON-113667225 tempest-AttachInterfacesTestJSON-113667225-project-member] [instance: ad58bbc3-1ec8-4567-ba07-c8161bcc8380] Updating instance_info_cache with network_info: [] {{(pid=52243) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 736.516367] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 736.540025] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 736.540025] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 736.540025] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 736.584949] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 736.584949] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 736.584949] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 736.584949] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 736.584949] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 736.584949] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 736.614627] nova-conductor[52243]: DEBUG nova.quota [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Getting quotas for project 35f14b46451443d48ce9154de8a09045. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 736.618435] nova-conductor[52243]: DEBUG nova.quota [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Getting quotas for user cd26bdcbd8994834ba745c00358233a6 and project 35f14b46451443d48ce9154de8a09045. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 736.623078] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] [instance: 508c2a14-5f5b-4968-843a-1378d1c46e2f] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 736.623492] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 736.623816] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 736.624920] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 736.627794] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] [instance: 508c2a14-5f5b-4968-843a-1378d1c46e2f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 736.627794] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 736.627965] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 736.628109] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 736.641731] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 736.641731] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 736.642184] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 736.716091] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: a411c5e7-5a49-463e-b270-800e35a31188] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port a63c96c0-e7e4-4bf2-9ea3-7ce0cbf6e40f, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance a411c5e7-5a49-463e-b270-800e35a31188 was re-scheduled: Binding failed for port a63c96c0-e7e4-4bf2-9ea3-7ce0cbf6e40f, please check neutron logs for more information.\n'] [ 736.716827] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 736.717121] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a411c5e7-5a49-463e-b270-800e35a31188.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a411c5e7-5a49-463e-b270-800e35a31188. [ 736.717398] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: a411c5e7-5a49-463e-b270-800e35a31188] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a411c5e7-5a49-463e-b270-800e35a31188. [ 736.755901] nova-conductor[52243]: DEBUG nova.network.neutron [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: a411c5e7-5a49-463e-b270-800e35a31188] deallocate_for_instance() {{(pid=52243) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 736.944930] nova-conductor[52243]: DEBUG nova.network.neutron [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: a411c5e7-5a49-463e-b270-800e35a31188] Instance cache missing network info. {{(pid=52243) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 736.977544] nova-conductor[52243]: DEBUG nova.network.neutron [None req-07b21824-c498-423f-91e4-177f5a80294a tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: a411c5e7-5a49-463e-b270-800e35a31188] Updating instance_info_cache with network_info: [] {{(pid=52243) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 737.022941] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] [instance: 1670e7a3-656a-444d-85ed-292956498612] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port ef1cd5c0-1945-454d-a3f4-40d46a56364f, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 1670e7a3-656a-444d-85ed-292956498612 was re-scheduled: Binding failed for port ef1cd5c0-1945-454d-a3f4-40d46a56364f, please check neutron logs for more information.\n'] [ 737.023589] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 737.023827] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 1670e7a3-656a-444d-85ed-292956498612.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 1670e7a3-656a-444d-85ed-292956498612. [ 737.024051] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] [instance: 1670e7a3-656a-444d-85ed-292956498612] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 1670e7a3-656a-444d-85ed-292956498612. [ 737.054400] nova-conductor[52243]: DEBUG nova.network.neutron [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] [instance: 1670e7a3-656a-444d-85ed-292956498612] deallocate_for_instance() {{(pid=52243) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 737.132659] nova-conductor[52243]: DEBUG nova.network.neutron [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] [instance: 1670e7a3-656a-444d-85ed-292956498612] Instance cache missing network info. {{(pid=52243) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 737.134975] nova-conductor[52243]: DEBUG nova.network.neutron [None req-4eb49f41-f4bb-489a-b035-264871b492e1 tempest-ServerRescueTestJSONUnderV235-580767535 tempest-ServerRescueTestJSONUnderV235-580767535-project-member] [instance: 1670e7a3-656a-444d-85ed-292956498612] Updating instance_info_cache with network_info: [] {{(pid=52243) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 740.355049] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Took 0.15 seconds to select destinations for 1 instance(s). {{(pid=52244) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 740.369979] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 740.370229] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 740.370424] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 740.402841] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 740.403084] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 740.403655] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 740.403655] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 740.403826] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 740.403975] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 740.415109] nova-conductor[52244]: DEBUG nova.quota [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Getting quotas for project 8689e7ba4d544dfcbbdf7c864cb3f823. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 740.418116] nova-conductor[52244]: DEBUG nova.quota [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Getting quotas for user 8090dd0e116e4ac89aeb07e25bc22927 and project 8689e7ba4d544dfcbbdf7c864cb3f823. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 740.424779] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: b8404801-b787-4db2-aa13-320f87ca5ac5] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52244) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 740.425366] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 740.425596] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 740.425861] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 740.429442] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: b8404801-b787-4db2-aa13-320f87ca5ac5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 740.429442] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 740.429663] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 740.429941] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 740.446855] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Took 0.13 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 740.458665] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 740.458884] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 740.462413] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 740.462647] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 740.462817] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 740.463305] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 740.504936] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 740.504936] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 740.505168] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 740.505823] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 740.505823] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 740.505823] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 740.516742] nova-conductor[52243]: DEBUG nova.quota [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Getting quotas for project 18959b240e3b49d6a94dc7aa1e92487c. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 740.518647] nova-conductor[52243]: DEBUG nova.quota [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Getting quotas for user d13e7b6d63fc4aa4aaf5c066c9cbfcbf and project 18959b240e3b49d6a94dc7aa1e92487c. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 740.524456] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] [instance: e2620c94-5629-4094-a92f-d83d9efd6205] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 740.524935] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 740.525200] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 740.525392] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 740.528542] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] [instance: e2620c94-5629-4094-a92f-d83d9efd6205] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 740.529217] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 740.529443] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 740.529704] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 740.546357] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 740.546357] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 740.547251] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 740.719580] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] [instance: ee7de02d-e1c1-4fe7-8df9-e9f82a39ef24] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 54f30955-59af-40cf-b87d-43e5cf498d0d, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ee7de02d-e1c1-4fe7-8df9-e9f82a39ef24 was re-scheduled: Binding failed for port 54f30955-59af-40cf-b87d-43e5cf498d0d, please check neutron logs for more information.\n'] [ 740.720238] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 740.720477] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ee7de02d-e1c1-4fe7-8df9-e9f82a39ef24.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ee7de02d-e1c1-4fe7-8df9-e9f82a39ef24. [ 740.720792] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] [instance: ee7de02d-e1c1-4fe7-8df9-e9f82a39ef24] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ee7de02d-e1c1-4fe7-8df9-e9f82a39ef24. [ 740.742196] nova-conductor[52243]: DEBUG nova.network.neutron [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] [instance: ee7de02d-e1c1-4fe7-8df9-e9f82a39ef24] deallocate_for_instance() {{(pid=52243) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 740.798317] nova-conductor[52243]: DEBUG nova.network.neutron [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] [instance: ee7de02d-e1c1-4fe7-8df9-e9f82a39ef24] Instance cache missing network info. {{(pid=52243) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 740.802254] nova-conductor[52243]: DEBUG nova.network.neutron [None req-00ef82e1-26e6-453a-86ca-1f086efe3204 tempest-ServerMetadataTestJSON-382877116 tempest-ServerMetadataTestJSON-382877116-project-member] [instance: ee7de02d-e1c1-4fe7-8df9-e9f82a39ef24] Updating instance_info_cache with network_info: [] {{(pid=52243) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 742.158432] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=52244) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 742.174434] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.174868] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.175092] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.208903] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 742.216295] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.216537] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.216612] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.217431] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.217597] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.217776] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.228038] nova-conductor[52244]: DEBUG nova.quota [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Getting quotas for project 7a9e45c3a1d148e7b494e0291dae92bf. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 742.230343] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.230726] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.231721] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.231818] nova-conductor[52244]: DEBUG nova.quota [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Getting quotas for user 4432ca36538f4b16b4cf842c7e286271 and project 7a9e45c3a1d148e7b494e0291dae92bf. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 742.243141] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] [instance: b4a2d6ae-f580-43a6-b1b7-9a4727c3ac30] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52244) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 742.243695] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.246547] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.246924] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.253358] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] [instance: b4a2d6ae-f580-43a6-b1b7-9a4727c3ac30] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 742.256046] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.256046] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.256046] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.268584] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.269042] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.269359] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.269867] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.270988] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.270988] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.280744] nova-conductor[52243]: DEBUG nova.quota [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Getting quotas for project dd2add382fb34e309cc9b0acd9403ef6. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 742.281577] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.281838] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.282013] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.283718] nova-conductor[52243]: DEBUG nova.quota [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Getting quotas for user 7c2b5f71e3034e5f90220c5ebf1bb6d9 and project dd2add382fb34e309cc9b0acd9403ef6. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 742.290450] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: f05a805d-7896-477c-b2ea-437faec88fba] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 742.291067] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.291384] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.291705] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.295262] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: f05a805d-7896-477c-b2ea-437faec88fba] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 742.296026] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.296255] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.296438] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.312580] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.312815] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.312982] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.559654] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=52244) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 742.581613] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.581779] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.581947] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.652550] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.652550] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.652550] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.652550] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.652550] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.652550] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.663055] nova-conductor[52244]: DEBUG nova.quota [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Getting quotas for project 352b723cfbb34bfa9e4500f104c508f1. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 742.665719] nova-conductor[52244]: DEBUG nova.quota [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Getting quotas for user c29eecb08be34b7b8e02ebeb75bf881c and project 352b723cfbb34bfa9e4500f104c508f1. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 742.679614] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] [instance: 78ed17da-e8e8-4872-b1bf-95c4e77de8e6] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52244) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 742.680206] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.680477] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.680664] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.685043] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] [instance: 78ed17da-e8e8-4872-b1bf-95c4e77de8e6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 742.685745] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.685957] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.686141] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.712473] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.713130] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.713130] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 744.678825] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] [instance: e90ee443-efe0-4f3e-999b-b9376e41fcb5] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 48ed80ba-0203-40d7-adfe-9f69bcfe2e45, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance e90ee443-efe0-4f3e-999b-b9376e41fcb5 was re-scheduled: Binding failed for port 48ed80ba-0203-40d7-adfe-9f69bcfe2e45, please check neutron logs for more information.\n'] [ 744.679472] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 744.679704] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e90ee443-efe0-4f3e-999b-b9376e41fcb5.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e90ee443-efe0-4f3e-999b-b9376e41fcb5. [ 744.682854] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] [instance: e90ee443-efe0-4f3e-999b-b9376e41fcb5] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e90ee443-efe0-4f3e-999b-b9376e41fcb5. [ 744.713250] nova-conductor[52244]: DEBUG nova.network.neutron [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] [instance: e90ee443-efe0-4f3e-999b-b9376e41fcb5] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 744.897200] nova-conductor[52244]: DEBUG nova.network.neutron [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] [instance: e90ee443-efe0-4f3e-999b-b9376e41fcb5] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 744.900552] nova-conductor[52244]: DEBUG nova.network.neutron [None req-a507a8ca-f2f0-480c-9b31-211a8ebd4d11 tempest-ServersTestJSON-1603309010 tempest-ServersTestJSON-1603309010-project-member] [instance: e90ee443-efe0-4f3e-999b-b9376e41fcb5] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 746.592098] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Took 0.15 seconds to select destinations for 1 instance(s). {{(pid=52244) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 746.603780] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 746.604040] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 746.604212] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 746.632457] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 746.632457] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 746.632746] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 746.633733] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 746.633733] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 746.633733] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 746.647959] nova-conductor[52244]: DEBUG nova.quota [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Getting quotas for project 91232108c8944a3da00233e9c54c9749. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 746.651040] nova-conductor[52244]: DEBUG nova.quota [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Getting quotas for user 593629524d524ad9b515d92b36e7b1e8 and project 91232108c8944a3da00233e9c54c9749. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 746.658139] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: abec9f87-4cde-4b5e-ad2a-fa682842ac7a] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52244) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 746.658692] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 746.659477] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 746.659477] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 746.663533] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: abec9f87-4cde-4b5e-ad2a-fa682842ac7a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 746.663667] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 746.664540] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 746.664540] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 746.680490] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 746.680653] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 746.680817] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 747.625978] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Took 0.14 seconds to select destinations for 1 instance(s). {{(pid=52244) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 747.638807] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 747.639207] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 747.639317] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 747.670825] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 747.671148] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 747.671333] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 747.671680] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 747.671867] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 747.672035] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 747.681849] nova-conductor[52244]: DEBUG nova.quota [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Getting quotas for project 352b723cfbb34bfa9e4500f104c508f1. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 747.684534] nova-conductor[52244]: DEBUG nova.quota [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Getting quotas for user c29eecb08be34b7b8e02ebeb75bf881c and project 352b723cfbb34bfa9e4500f104c508f1. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 747.690698] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] [instance: a75a3491-94b0-4754-8e42-7bf49194a022] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52244) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 747.691035] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 747.691239] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 747.691402] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 747.695445] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] [instance: a75a3491-94b0-4754-8e42-7bf49194a022] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 747.695889] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 747.696105] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 747.696298] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 747.709075] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 747.709286] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 747.709554] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 750.432311] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] [instance: 508c2a14-5f5b-4968-843a-1378d1c46e2f] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 45008a60-2dcb-40dd-b9e0-f4d0e6137628, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 508c2a14-5f5b-4968-843a-1378d1c46e2f was re-scheduled: Binding failed for port 45008a60-2dcb-40dd-b9e0-f4d0e6137628, please check neutron logs for more information.\n'] [ 750.433211] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 750.433494] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 508c2a14-5f5b-4968-843a-1378d1c46e2f.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 508c2a14-5f5b-4968-843a-1378d1c46e2f. [ 750.433843] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] [instance: 508c2a14-5f5b-4968-843a-1378d1c46e2f] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 508c2a14-5f5b-4968-843a-1378d1c46e2f. [ 750.456115] nova-conductor[52243]: DEBUG nova.network.neutron [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] [instance: 508c2a14-5f5b-4968-843a-1378d1c46e2f] deallocate_for_instance() {{(pid=52243) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 750.553632] nova-conductor[52243]: DEBUG nova.network.neutron [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] [instance: 508c2a14-5f5b-4968-843a-1378d1c46e2f] Instance cache missing network info. {{(pid=52243) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 750.561018] nova-conductor[52243]: DEBUG nova.network.neutron [None req-54d6db5f-fa9c-4c6d-83d6-e41dfdf83fa0 tempest-AttachVolumeTestJSON-1478151191 tempest-AttachVolumeTestJSON-1478151191-project-member] [instance: 508c2a14-5f5b-4968-843a-1378d1c46e2f] Updating instance_info_cache with network_info: [] {{(pid=52243) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 753.310525] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] [instance: e2620c94-5629-4094-a92f-d83d9efd6205] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b97ea0a9-00ea-409a-ba57-0c68616a1f0a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance e2620c94-5629-4094-a92f-d83d9efd6205 was re-scheduled: Binding failed for port b97ea0a9-00ea-409a-ba57-0c68616a1f0a, please check neutron logs for more information.\n'] [ 753.312522] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 753.312522] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e2620c94-5629-4094-a92f-d83d9efd6205.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e2620c94-5629-4094-a92f-d83d9efd6205. [ 753.312522] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] [instance: e2620c94-5629-4094-a92f-d83d9efd6205] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e2620c94-5629-4094-a92f-d83d9efd6205. [ 753.344860] nova-conductor[52243]: DEBUG nova.network.neutron [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] [instance: e2620c94-5629-4094-a92f-d83d9efd6205] deallocate_for_instance() {{(pid=52243) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 753.434227] nova-conductor[52243]: DEBUG nova.network.neutron [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] [instance: e2620c94-5629-4094-a92f-d83d9efd6205] Instance cache missing network info. {{(pid=52243) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 753.438539] nova-conductor[52243]: DEBUG nova.network.neutron [None req-feb153f7-953e-4e96-b1a6-23f5996cad56 tempest-AttachVolumeShelveTestJSON-687008755 tempest-AttachVolumeShelveTestJSON-687008755-project-member] [instance: e2620c94-5629-4094-a92f-d83d9efd6205] Updating instance_info_cache with network_info: [] {{(pid=52243) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 754.079510] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: b8404801-b787-4db2-aa13-320f87ca5ac5] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port b6e44e9f-1e71-4450-80a0-4d9203c470e9, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance b8404801-b787-4db2-aa13-320f87ca5ac5 was re-scheduled: Binding failed for port b6e44e9f-1e71-4450-80a0-4d9203c470e9, please check neutron logs for more information.\n'] [ 754.080721] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 754.080721] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b8404801-b787-4db2-aa13-320f87ca5ac5.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b8404801-b787-4db2-aa13-320f87ca5ac5. [ 754.080721] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: b8404801-b787-4db2-aa13-320f87ca5ac5] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b8404801-b787-4db2-aa13-320f87ca5ac5. [ 754.116934] nova-conductor[52244]: DEBUG nova.network.neutron [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: b8404801-b787-4db2-aa13-320f87ca5ac5] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 754.272294] nova-conductor[52244]: DEBUG nova.network.neutron [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: b8404801-b787-4db2-aa13-320f87ca5ac5] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 754.279018] nova-conductor[52244]: DEBUG nova.network.neutron [None req-3b83ff7a-707a-4d4d-9e7a-aa78aae545c9 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: b8404801-b787-4db2-aa13-320f87ca5ac5] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 754.546085] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Took 0.14 seconds to select destinations for 1 instance(s). {{(pid=52244) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 754.559958] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 754.560332] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 754.560567] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 754.591188] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 754.591496] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 754.591737] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 754.592166] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 754.592411] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 754.592625] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 754.602028] nova-conductor[52244]: DEBUG nova.quota [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Getting quotas for project 145cbf0470004e0e8f468d7823433a21. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 754.605790] nova-conductor[52244]: DEBUG nova.quota [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Getting quotas for user 88574e74047e494a8e13911db17cffc3 and project 145cbf0470004e0e8f468d7823433a21. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 754.614096] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] [instance: 1195d592-faa2-43d7-af58-12b75abd5ed0] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52244) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 754.614578] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 754.614753] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 754.614922] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 754.620076] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] [instance: 1195d592-faa2-43d7-af58-12b75abd5ed0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 754.620532] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 754.620737] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 754.620903] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 754.633506] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 754.633717] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 754.633884] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 756.799120] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Took 0.18 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 756.814360] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 756.815595] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 756.815699] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 756.869125] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 756.869658] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 756.869658] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 756.869935] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 756.870078] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 756.870233] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 756.883940] nova-conductor[52243]: DEBUG nova.quota [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Getting quotas for project f8b200f828624b43b290c7894018dec6. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 756.886561] nova-conductor[52243]: DEBUG nova.quota [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Getting quotas for user 94b356ff05594435a3bde51ddccf333b and project f8b200f828624b43b290c7894018dec6. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 756.893491] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] [instance: 0ed4be35-b845-48ca-b892-657d96c12728] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 756.893938] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 756.894159] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 756.894350] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 756.900566] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] [instance: 0ed4be35-b845-48ca-b892-657d96c12728] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 756.902486] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 756.902696] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 756.902857] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 756.922802] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 756.923038] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 756.923216] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 757.640592] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=52244) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 757.661657] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 757.661882] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 757.662085] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 757.722113] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 757.722366] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 757.722543] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 757.722946] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 757.723095] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 757.723257] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 757.737507] nova-conductor[52244]: DEBUG nova.quota [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Getting quotas for project 392af2151280472aa22ca8faa2f253ce. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 757.744723] nova-conductor[52244]: DEBUG nova.quota [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Getting quotas for user 56b440aff52c4608a2db2ddbe818847b and project 392af2151280472aa22ca8faa2f253ce. Resources: {'ram', 'cores', 'instances'} {{(pid=52244) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 757.750790] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] [instance: 938a2016-8eaa-446a-b69c-3af59448d944] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52244) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 757.750790] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 757.750790] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 757.751571] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 757.758214] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] [instance: 938a2016-8eaa-446a-b69c-3af59448d944] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 757.758214] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 757.758214] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 757.758214] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 757.780314] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 757.780314] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 757.780314] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 758.604794] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] [instance: b4a2d6ae-f580-43a6-b1b7-9a4727c3ac30] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 54e48d56-6f7b-4965-aed4-4439616ed0bf, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance b4a2d6ae-f580-43a6-b1b7-9a4727c3ac30 was re-scheduled: Binding failed for port 54e48d56-6f7b-4965-aed4-4439616ed0bf, please check neutron logs for more information.\n'] [ 758.604794] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 758.604794] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b4a2d6ae-f580-43a6-b1b7-9a4727c3ac30.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b4a2d6ae-f580-43a6-b1b7-9a4727c3ac30. [ 758.604794] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] [instance: b4a2d6ae-f580-43a6-b1b7-9a4727c3ac30] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b4a2d6ae-f580-43a6-b1b7-9a4727c3ac30. [ 758.638198] nova-conductor[52243]: DEBUG nova.network.neutron [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] [instance: b4a2d6ae-f580-43a6-b1b7-9a4727c3ac30] deallocate_for_instance() {{(pid=52243) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 758.666269] nova-conductor[52243]: DEBUG nova.network.neutron [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] [instance: b4a2d6ae-f580-43a6-b1b7-9a4727c3ac30] Instance cache missing network info. {{(pid=52243) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 758.671352] nova-conductor[52243]: DEBUG nova.network.neutron [None req-6e21e153-a5d9-418f-91b1-946ea426e46d tempest-FloatingIPsAssociationTestJSON-246739974 tempest-FloatingIPsAssociationTestJSON-246739974-project-member] [instance: b4a2d6ae-f580-43a6-b1b7-9a4727c3ac30] Updating instance_info_cache with network_info: [] {{(pid=52243) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 758.763966] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Took 0.14 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 758.779554] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 758.779975] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 758.779975] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 758.812825] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 758.813067] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 758.813220] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 758.813566] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 758.813742] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 758.814462] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 758.829959] nova-conductor[52243]: DEBUG nova.quota [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Getting quotas for project f858ee4b23fb49c399f103c4a8bcdebc. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 758.833316] nova-conductor[52243]: DEBUG nova.quota [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Getting quotas for user b8d25c6dcda1421b82c920c9580bf020 and project f858ee4b23fb49c399f103c4a8bcdebc. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 758.838912] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] [instance: 72a92098-562e-47bf-8dde-8b62b182d7bb] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 758.839373] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 758.839574] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 758.839733] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 758.844293] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] [instance: 72a92098-562e-47bf-8dde-8b62b182d7bb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 758.844862] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 758.845084] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 758.845310] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 758.857855] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 758.858110] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 758.858291] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 759.057259] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: f05a805d-7896-477c-b2ea-437faec88fba] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 96439e30-308b-4bba-82b1-71b05db60ec7, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance f05a805d-7896-477c-b2ea-437faec88fba was re-scheduled: Binding failed for port 96439e30-308b-4bba-82b1-71b05db60ec7, please check neutron logs for more information.\n'] [ 759.058423] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 759.058787] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f05a805d-7896-477c-b2ea-437faec88fba.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f05a805d-7896-477c-b2ea-437faec88fba. [ 759.061964] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: f05a805d-7896-477c-b2ea-437faec88fba] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f05a805d-7896-477c-b2ea-437faec88fba. [ 759.087110] nova-conductor[52243]: DEBUG nova.network.neutron [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: f05a805d-7896-477c-b2ea-437faec88fba] deallocate_for_instance() {{(pid=52243) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 759.150730] nova-conductor[52243]: DEBUG nova.network.neutron [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: f05a805d-7896-477c-b2ea-437faec88fba] Instance cache missing network info. {{(pid=52243) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 759.155147] nova-conductor[52243]: DEBUG nova.network.neutron [None req-20a0ea14-6eb7-4fc0-84ed-491fc82e1ed9 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: f05a805d-7896-477c-b2ea-437faec88fba] Updating instance_info_cache with network_info: [] {{(pid=52243) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 760.183549] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 760.201657] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 760.201897] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 760.202082] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 760.234120] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 760.234120] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 760.234120] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 760.234419] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 760.234795] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 760.234894] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 760.246528] nova-conductor[52243]: DEBUG nova.quota [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Getting quotas for project f8f74d0c25be4992b4701c8838638ab3. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 760.249052] nova-conductor[52243]: DEBUG nova.quota [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Getting quotas for user 92c8ce085a514809aaf0fabb9982ccd2 and project f8f74d0c25be4992b4701c8838638ab3. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 760.259010] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] [instance: 2d63a2a4-b912-487e-aa10-9e68d877baab] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 760.259642] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 760.259755] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 760.259953] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 760.264217] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] [instance: 2d63a2a4-b912-487e-aa10-9e68d877baab] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 760.265430] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 760.266028] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 760.266028] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 760.282042] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 760.282279] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 760.282448] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 761.038573] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] [instance: 1195d592-faa2-43d7-af58-12b75abd5ed0] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 8ff4da7c-e6f1-47e2-8c4b-f5abfbc23ffa, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 1195d592-faa2-43d7-af58-12b75abd5ed0 was re-scheduled: Binding failed for port 8ff4da7c-e6f1-47e2-8c4b-f5abfbc23ffa, please check neutron logs for more information.\n'] [ 761.039394] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 761.039633] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 1195d592-faa2-43d7-af58-12b75abd5ed0.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 1195d592-faa2-43d7-af58-12b75abd5ed0. [ 761.039949] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] [instance: 1195d592-faa2-43d7-af58-12b75abd5ed0] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 1195d592-faa2-43d7-af58-12b75abd5ed0. [ 761.073636] nova-conductor[52244]: DEBUG nova.network.neutron [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] [instance: 1195d592-faa2-43d7-af58-12b75abd5ed0] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 761.221383] nova-conductor[52244]: DEBUG nova.network.neutron [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] [instance: 1195d592-faa2-43d7-af58-12b75abd5ed0] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 761.226107] nova-conductor[52244]: DEBUG nova.network.neutron [None req-8908f05b-872f-4666-993f-86042f4b27f5 tempest-ServerActionsTestJSON-335386642 tempest-ServerActionsTestJSON-335386642-project-member] [instance: 1195d592-faa2-43d7-af58-12b75abd5ed0] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 762.418494] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: abec9f87-4cde-4b5e-ad2a-fa682842ac7a] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port f3f7985f-2ffb-489b-bf50-5a30759b413b, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance abec9f87-4cde-4b5e-ad2a-fa682842ac7a was re-scheduled: Binding failed for port f3f7985f-2ffb-489b-bf50-5a30759b413b, please check neutron logs for more information.\n'] [ 762.418878] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 762.418878] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance abec9f87-4cde-4b5e-ad2a-fa682842ac7a.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance abec9f87-4cde-4b5e-ad2a-fa682842ac7a. [ 762.419362] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: abec9f87-4cde-4b5e-ad2a-fa682842ac7a] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance abec9f87-4cde-4b5e-ad2a-fa682842ac7a. [ 762.451909] nova-conductor[52243]: DEBUG nova.network.neutron [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: abec9f87-4cde-4b5e-ad2a-fa682842ac7a] deallocate_for_instance() {{(pid=52243) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 762.565343] nova-conductor[52243]: DEBUG nova.network.neutron [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: abec9f87-4cde-4b5e-ad2a-fa682842ac7a] Instance cache missing network info. {{(pid=52243) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 762.569365] nova-conductor[52243]: DEBUG nova.network.neutron [None req-77dcc848-a5f0-4c0e-92f4-b129eec2903d tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: abec9f87-4cde-4b5e-ad2a-fa682842ac7a] Updating instance_info_cache with network_info: [] {{(pid=52243) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 762.955811] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Took 0.15 seconds to select destinations for 1 instance(s). {{(pid=52243) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:945}} [ 762.989743] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] [instance: ea968312-62ea-4f55-87e9-f91823fc14c2] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance ea968312-62ea-4f55-87e9-f91823fc14c2 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 762.990382] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 762.991209] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ea968312-62ea-4f55-87e9-f91823fc14c2.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ea968312-62ea-4f55-87e9-f91823fc14c2. [ 762.991209] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-20524171-0491-451c-acd6-bf4548e093fb tempest-ServerShowV257Test-1565272682 tempest-ServerShowV257Test-1565272682-project-member] [instance: ea968312-62ea-4f55-87e9-f91823fc14c2] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ea968312-62ea-4f55-87e9-f91823fc14c2. [ 763.009789] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 763.010074] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 763.010293] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 763.054649] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 763.054880] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 763.055061] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 763.055429] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 763.055668] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 763.055836] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 763.063742] nova-conductor[52243]: DEBUG nova.quota [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Getting quotas for project d099026ef14c4a9ea51a62cb8dc15673. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 763.068679] nova-conductor[52243]: DEBUG nova.quota [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Getting quotas for user cd62aa2126d64e81a565bc0d29c7f13d and project d099026ef14c4a9ea51a62cb8dc15673. Resources: {'ram', 'cores', 'instances'} {{(pid=52243) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 763.074574] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] [instance: f31b7ca3-60d1-4206-ac49-f85ec6194f85] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52243) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1761}} [ 763.075017] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 763.075226] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 763.075483] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 763.078264] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] [instance: f31b7ca3-60d1-4206-ac49-f85ec6194f85] block_device_mapping [BlockDeviceMapping(attachment_id=494fa8ca-c212-4820-824a-b6c7e29bbe82,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='edd23c41-bc61-47c0-9e80-412513b26303',volume_size=1,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 763.078909] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 763.079121] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 763.079287] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 763.094825] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 763.094900] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 763.095088] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 764.370456] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] [instance: 2d63a2a4-b912-487e-aa10-9e68d877baab] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 7f2b0be1-4d42-4abc-a751-d4b312b5c4a5, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 2d63a2a4-b912-487e-aa10-9e68d877baab was re-scheduled: Binding failed for port 7f2b0be1-4d42-4abc-a751-d4b312b5c4a5, please check neutron logs for more information.\n'] [ 764.371179] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 764.371468] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2d63a2a4-b912-487e-aa10-9e68d877baab.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2d63a2a4-b912-487e-aa10-9e68d877baab. [ 764.371957] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] [instance: 2d63a2a4-b912-487e-aa10-9e68d877baab] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2d63a2a4-b912-487e-aa10-9e68d877baab. [ 764.398274] nova-conductor[52244]: DEBUG nova.network.neutron [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] [instance: 2d63a2a4-b912-487e-aa10-9e68d877baab] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 764.494924] nova-conductor[52244]: DEBUG nova.network.neutron [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] [instance: 2d63a2a4-b912-487e-aa10-9e68d877baab] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 764.499194] nova-conductor[52244]: DEBUG nova.network.neutron [None req-57ce834c-bcce-426a-ba4b-ceda0f1c5b75 tempest-ServersTestFqdnHostnames-512619947 tempest-ServersTestFqdnHostnames-512619947-project-member] [instance: 2d63a2a4-b912-487e-aa10-9e68d877baab] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager [None req-96b42d87-98f9-4721-895e-37005f320fb3 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 764.626252] nova-conductor[52244]: Traceback (most recent call last): [ 764.626252] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 764.626252] nova-conductor[52244]: return func(*args, **kwargs) [ 764.626252] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 764.626252] nova-conductor[52244]: selections = self._select_destinations( [ 764.626252] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 764.626252] nova-conductor[52244]: selections = self._schedule( [ 764.626252] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 764.626252] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 764.626252] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 764.626252] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 764.626252] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager [ 764.626252] nova-conductor[52244]: ERROR nova.conductor.manager [ 764.635639] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-96b42d87-98f9-4721-895e-37005f320fb3 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 764.635850] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-96b42d87-98f9-4721-895e-37005f320fb3 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 764.636039] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-96b42d87-98f9-4721-895e-37005f320fb3 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager [None req-4a56b9bb-27ff-4b74-9662-940e85dcfd99 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 764.652238] nova-conductor[52243]: Traceback (most recent call last): [ 764.652238] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 764.652238] nova-conductor[52243]: return func(*args, **kwargs) [ 764.652238] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 764.652238] nova-conductor[52243]: selections = self._select_destinations( [ 764.652238] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 764.652238] nova-conductor[52243]: selections = self._schedule( [ 764.652238] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 764.652238] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 764.652238] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 764.652238] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 764.652238] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager [ 764.652238] nova-conductor[52243]: ERROR nova.conductor.manager [ 764.660031] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4a56b9bb-27ff-4b74-9662-940e85dcfd99 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 764.660250] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4a56b9bb-27ff-4b74-9662-940e85dcfd99 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 764.660425] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4a56b9bb-27ff-4b74-9662-940e85dcfd99 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 764.683189] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-96b42d87-98f9-4721-895e-37005f320fb3 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: 563c0d3b-c08c-484b-80b2-7e029ccc0a04] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 764.683883] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-96b42d87-98f9-4721-895e-37005f320fb3 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 764.684109] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-96b42d87-98f9-4721-895e-37005f320fb3 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 764.684281] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-96b42d87-98f9-4721-895e-37005f320fb3 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 764.687224] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-96b42d87-98f9-4721-895e-37005f320fb3 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 764.687224] nova-conductor[52244]: Traceback (most recent call last): [ 764.687224] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 764.687224] nova-conductor[52244]: return func(*args, **kwargs) [ 764.687224] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 764.687224] nova-conductor[52244]: selections = self._select_destinations( [ 764.687224] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 764.687224] nova-conductor[52244]: selections = self._schedule( [ 764.687224] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 764.687224] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 764.687224] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 764.687224] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 764.687224] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 764.687224] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 764.687730] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-96b42d87-98f9-4721-895e-37005f320fb3 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: 563c0d3b-c08c-484b-80b2-7e029ccc0a04] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 764.700818] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-4a56b9bb-27ff-4b74-9662-940e85dcfd99 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] [instance: 4b971377-36b2-4512-8aad-2684e8f32c80] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 764.701560] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4a56b9bb-27ff-4b74-9662-940e85dcfd99 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 764.701722] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4a56b9bb-27ff-4b74-9662-940e85dcfd99 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 764.701894] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4a56b9bb-27ff-4b74-9662-940e85dcfd99 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 764.707028] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-4a56b9bb-27ff-4b74-9662-940e85dcfd99 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 764.707028] nova-conductor[52243]: Traceback (most recent call last): [ 764.707028] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 764.707028] nova-conductor[52243]: return func(*args, **kwargs) [ 764.707028] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 764.707028] nova-conductor[52243]: selections = self._select_destinations( [ 764.707028] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 764.707028] nova-conductor[52243]: selections = self._schedule( [ 764.707028] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 764.707028] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 764.707028] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 764.707028] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 764.707028] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 764.707028] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 764.707679] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-4a56b9bb-27ff-4b74-9662-940e85dcfd99 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] [instance: 4b971377-36b2-4512-8aad-2684e8f32c80] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 765.975348] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] [instance: f31b7ca3-60d1-4206-ac49-f85ec6194f85] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port db49ee90-852b-4303-8959-d6a2ac7694bd, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance f31b7ca3-60d1-4206-ac49-f85ec6194f85 was re-scheduled: Binding failed for port db49ee90-852b-4303-8959-d6a2ac7694bd, please check neutron logs for more information.\n'] [ 765.975905] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 765.976147] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f31b7ca3-60d1-4206-ac49-f85ec6194f85.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f31b7ca3-60d1-4206-ac49-f85ec6194f85. [ 765.976364] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] [instance: f31b7ca3-60d1-4206-ac49-f85ec6194f85] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f31b7ca3-60d1-4206-ac49-f85ec6194f85. [ 766.006942] nova-conductor[52243]: DEBUG nova.network.neutron [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] [instance: f31b7ca3-60d1-4206-ac49-f85ec6194f85] deallocate_for_instance() {{(pid=52243) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 766.028313] nova-conductor[52243]: DEBUG nova.network.neutron [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] [instance: f31b7ca3-60d1-4206-ac49-f85ec6194f85] Instance cache missing network info. {{(pid=52243) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 766.032458] nova-conductor[52243]: DEBUG nova.network.neutron [None req-6c2ffaa1-e526-4f8f-8dd1-68bec88e766a tempest-ServerActionsV293TestJSON-1756577926 tempest-ServerActionsV293TestJSON-1756577926-project-member] [instance: f31b7ca3-60d1-4206-ac49-f85ec6194f85] Updating instance_info_cache with network_info: [] {{(pid=52243) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager [None req-a7361e93-5a3a-4afb-b48b-9a3780f01b4b tempest-ServerPasswordTestJSON-443041768 tempest-ServerPasswordTestJSON-443041768-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 767.575220] nova-conductor[52243]: Traceback (most recent call last): [ 767.575220] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 767.575220] nova-conductor[52243]: return func(*args, **kwargs) [ 767.575220] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 767.575220] nova-conductor[52243]: selections = self._select_destinations( [ 767.575220] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 767.575220] nova-conductor[52243]: selections = self._schedule( [ 767.575220] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 767.575220] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 767.575220] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 767.575220] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 767.575220] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager [ 767.575220] nova-conductor[52243]: ERROR nova.conductor.manager [ 767.583599] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a7361e93-5a3a-4afb-b48b-9a3780f01b4b tempest-ServerPasswordTestJSON-443041768 tempest-ServerPasswordTestJSON-443041768-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 767.583899] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a7361e93-5a3a-4afb-b48b-9a3780f01b4b tempest-ServerPasswordTestJSON-443041768 tempest-ServerPasswordTestJSON-443041768-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 767.584130] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a7361e93-5a3a-4afb-b48b-9a3780f01b4b tempest-ServerPasswordTestJSON-443041768 tempest-ServerPasswordTestJSON-443041768-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 767.638031] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-a7361e93-5a3a-4afb-b48b-9a3780f01b4b tempest-ServerPasswordTestJSON-443041768 tempest-ServerPasswordTestJSON-443041768-project-member] [instance: 98e71cf6-94e8-4107-8d61-bfa799620fc9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 767.639027] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a7361e93-5a3a-4afb-b48b-9a3780f01b4b tempest-ServerPasswordTestJSON-443041768 tempest-ServerPasswordTestJSON-443041768-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 767.639027] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a7361e93-5a3a-4afb-b48b-9a3780f01b4b tempest-ServerPasswordTestJSON-443041768 tempest-ServerPasswordTestJSON-443041768-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 767.639027] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a7361e93-5a3a-4afb-b48b-9a3780f01b4b tempest-ServerPasswordTestJSON-443041768 tempest-ServerPasswordTestJSON-443041768-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 767.642532] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-a7361e93-5a3a-4afb-b48b-9a3780f01b4b tempest-ServerPasswordTestJSON-443041768 tempest-ServerPasswordTestJSON-443041768-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 767.642532] nova-conductor[52243]: Traceback (most recent call last): [ 767.642532] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 767.642532] nova-conductor[52243]: return func(*args, **kwargs) [ 767.642532] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 767.642532] nova-conductor[52243]: selections = self._select_destinations( [ 767.642532] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 767.642532] nova-conductor[52243]: selections = self._schedule( [ 767.642532] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 767.642532] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 767.642532] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 767.642532] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 767.642532] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 767.642532] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 767.643076] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-a7361e93-5a3a-4afb-b48b-9a3780f01b4b tempest-ServerPasswordTestJSON-443041768 tempest-ServerPasswordTestJSON-443041768-project-member] [instance: 98e71cf6-94e8-4107-8d61-bfa799620fc9] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager [None req-5d7caa52-b712-4f57-a65c-c3f0fc61ca82 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 768.229677] nova-conductor[52244]: Traceback (most recent call last): [ 768.229677] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 768.229677] nova-conductor[52244]: return func(*args, **kwargs) [ 768.229677] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 768.229677] nova-conductor[52244]: selections = self._select_destinations( [ 768.229677] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 768.229677] nova-conductor[52244]: selections = self._schedule( [ 768.229677] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 768.229677] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 768.229677] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 768.229677] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 768.229677] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager [ 768.229677] nova-conductor[52244]: ERROR nova.conductor.manager [ 768.244401] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-5d7caa52-b712-4f57-a65c-c3f0fc61ca82 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 768.244658] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-5d7caa52-b712-4f57-a65c-c3f0fc61ca82 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 768.244845] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-5d7caa52-b712-4f57-a65c-c3f0fc61ca82 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 768.305662] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-5d7caa52-b712-4f57-a65c-c3f0fc61ca82 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] [instance: bbe8d0ce-a8c5-460d-812b-9e776f9a7721] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 768.306445] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-5d7caa52-b712-4f57-a65c-c3f0fc61ca82 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 768.306654] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-5d7caa52-b712-4f57-a65c-c3f0fc61ca82 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 768.306822] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-5d7caa52-b712-4f57-a65c-c3f0fc61ca82 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 768.310519] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-5d7caa52-b712-4f57-a65c-c3f0fc61ca82 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 768.310519] nova-conductor[52244]: Traceback (most recent call last): [ 768.310519] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 768.310519] nova-conductor[52244]: return func(*args, **kwargs) [ 768.310519] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 768.310519] nova-conductor[52244]: selections = self._select_destinations( [ 768.310519] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 768.310519] nova-conductor[52244]: selections = self._schedule( [ 768.310519] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 768.310519] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 768.310519] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 768.310519] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 768.310519] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 768.310519] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 768.311344] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-5d7caa52-b712-4f57-a65c-c3f0fc61ca82 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] [instance: bbe8d0ce-a8c5-460d-812b-9e776f9a7721] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager [None req-f143b18b-ffc2-468e-9068-e516efb33487 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 768.891803] nova-conductor[52243]: Traceback (most recent call last): [ 768.891803] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 768.891803] nova-conductor[52243]: return func(*args, **kwargs) [ 768.891803] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 768.891803] nova-conductor[52243]: selections = self._select_destinations( [ 768.891803] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 768.891803] nova-conductor[52243]: selections = self._schedule( [ 768.891803] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 768.891803] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 768.891803] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 768.891803] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 768.891803] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager [ 768.891803] nova-conductor[52243]: ERROR nova.conductor.manager [ 768.898721] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-f143b18b-ffc2-468e-9068-e516efb33487 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 768.899041] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-f143b18b-ffc2-468e-9068-e516efb33487 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 768.899331] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-f143b18b-ffc2-468e-9068-e516efb33487 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 768.959414] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-f143b18b-ffc2-468e-9068-e516efb33487 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: 5fd87270-2cb4-4b82-aac7-ce774fb9e704] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 768.960236] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-f143b18b-ffc2-468e-9068-e516efb33487 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 768.960520] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-f143b18b-ffc2-468e-9068-e516efb33487 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 768.960775] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-f143b18b-ffc2-468e-9068-e516efb33487 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 768.964455] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-f143b18b-ffc2-468e-9068-e516efb33487 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 768.964455] nova-conductor[52243]: Traceback (most recent call last): [ 768.964455] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 768.964455] nova-conductor[52243]: return func(*args, **kwargs) [ 768.964455] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 768.964455] nova-conductor[52243]: selections = self._select_destinations( [ 768.964455] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 768.964455] nova-conductor[52243]: selections = self._schedule( [ 768.964455] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 768.964455] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 768.964455] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 768.964455] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 768.964455] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 768.964455] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 768.965293] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-f143b18b-ffc2-468e-9068-e516efb33487 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: 5fd87270-2cb4-4b82-aac7-ce774fb9e704] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager [None req-76123a85-7758-42f2-95e2-0c0b650306cf tempest-InstanceActionsV221TestJSON-1108763158 tempest-InstanceActionsV221TestJSON-1108763158-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 770.079819] nova-conductor[52243]: Traceback (most recent call last): [ 770.079819] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 770.079819] nova-conductor[52243]: return func(*args, **kwargs) [ 770.079819] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 770.079819] nova-conductor[52243]: selections = self._select_destinations( [ 770.079819] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 770.079819] nova-conductor[52243]: selections = self._schedule( [ 770.079819] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 770.079819] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 770.079819] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 770.079819] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 770.079819] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager [ 770.079819] nova-conductor[52243]: ERROR nova.conductor.manager [ 770.090525] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-76123a85-7758-42f2-95e2-0c0b650306cf tempest-InstanceActionsV221TestJSON-1108763158 tempest-InstanceActionsV221TestJSON-1108763158-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 770.090766] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-76123a85-7758-42f2-95e2-0c0b650306cf tempest-InstanceActionsV221TestJSON-1108763158 tempest-InstanceActionsV221TestJSON-1108763158-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 770.090939] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-76123a85-7758-42f2-95e2-0c0b650306cf tempest-InstanceActionsV221TestJSON-1108763158 tempest-InstanceActionsV221TestJSON-1108763158-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 770.138560] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-76123a85-7758-42f2-95e2-0c0b650306cf tempest-InstanceActionsV221TestJSON-1108763158 tempest-InstanceActionsV221TestJSON-1108763158-project-member] [instance: 05feb163-b097-40bd-9889-7ca4479876c8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 770.139288] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-76123a85-7758-42f2-95e2-0c0b650306cf tempest-InstanceActionsV221TestJSON-1108763158 tempest-InstanceActionsV221TestJSON-1108763158-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 770.142622] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-76123a85-7758-42f2-95e2-0c0b650306cf tempest-InstanceActionsV221TestJSON-1108763158 tempest-InstanceActionsV221TestJSON-1108763158-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 770.142622] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-76123a85-7758-42f2-95e2-0c0b650306cf tempest-InstanceActionsV221TestJSON-1108763158 tempest-InstanceActionsV221TestJSON-1108763158-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 770.145924] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-76123a85-7758-42f2-95e2-0c0b650306cf tempest-InstanceActionsV221TestJSON-1108763158 tempest-InstanceActionsV221TestJSON-1108763158-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 770.145924] nova-conductor[52243]: Traceback (most recent call last): [ 770.145924] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 770.145924] nova-conductor[52243]: return func(*args, **kwargs) [ 770.145924] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 770.145924] nova-conductor[52243]: selections = self._select_destinations( [ 770.145924] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 770.145924] nova-conductor[52243]: selections = self._schedule( [ 770.145924] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 770.145924] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 770.145924] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 770.145924] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 770.145924] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 770.145924] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 770.146472] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-76123a85-7758-42f2-95e2-0c0b650306cf tempest-InstanceActionsV221TestJSON-1108763158 tempest-InstanceActionsV221TestJSON-1108763158-project-member] [instance: 05feb163-b097-40bd-9889-7ca4479876c8] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 770.570452] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] [instance: 0ed4be35-b845-48ca-b892-657d96c12728] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 560b15a6-3e21-4068-94d0-df0d6e201268, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 0ed4be35-b845-48ca-b892-657d96c12728 was re-scheduled: Binding failed for port 560b15a6-3e21-4068-94d0-df0d6e201268, please check neutron logs for more information.\n'] [ 770.570858] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 770.571040] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0ed4be35-b845-48ca-b892-657d96c12728.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0ed4be35-b845-48ca-b892-657d96c12728. [ 770.571236] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] [instance: 0ed4be35-b845-48ca-b892-657d96c12728] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 0ed4be35-b845-48ca-b892-657d96c12728. [ 770.595349] nova-conductor[52243]: DEBUG nova.network.neutron [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] [instance: 0ed4be35-b845-48ca-b892-657d96c12728] deallocate_for_instance() {{(pid=52243) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 770.678327] nova-conductor[52243]: DEBUG nova.network.neutron [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] [instance: 0ed4be35-b845-48ca-b892-657d96c12728] Instance cache missing network info. {{(pid=52243) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 770.694619] nova-conductor[52243]: DEBUG nova.network.neutron [None req-b211c439-48eb-45de-87a9-a1524f59bf9a tempest-ServerMetadataNegativeTestJSON-1677081655 tempest-ServerMetadataNegativeTestJSON-1677081655-project-member] [instance: 0ed4be35-b845-48ca-b892-657d96c12728] Updating instance_info_cache with network_info: [] {{(pid=52243) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 770.777488] nova-conductor[52243]: Traceback (most recent call last): [ 770.777488] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 770.777488] nova-conductor[52243]: return func(*args, **kwargs) [ 770.777488] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 770.777488] nova-conductor[52243]: selections = self._select_destinations( [ 770.777488] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 770.777488] nova-conductor[52243]: selections = self._schedule( [ 770.777488] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 770.777488] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 770.777488] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 770.777488] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 770.777488] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager [ 770.777488] nova-conductor[52243]: ERROR nova.conductor.manager [ 770.793529] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 770.794112] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 770.794112] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 770.820641] nova-conductor[52244]: ERROR nova.scheduler.utils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] [instance: 938a2016-8eaa-446a-b69c-3af59448d944] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 919aae49-1ef4-4d7f-a76b-82c6e8107512, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 938a2016-8eaa-446a-b69c-3af59448d944 was re-scheduled: Binding failed for port 919aae49-1ef4-4d7f-a76b-82c6e8107512, please check neutron logs for more information.\n'] [ 770.821772] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Rescheduling: True {{(pid=52244) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 770.821772] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 938a2016-8eaa-446a-b69c-3af59448d944.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 938a2016-8eaa-446a-b69c-3af59448d944. [ 770.822564] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] [instance: 938a2016-8eaa-446a-b69c-3af59448d944] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 938a2016-8eaa-446a-b69c-3af59448d944. [ 770.860710] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] [instance: 9f290d28-a018-4a06-8c84-81463a83b0b2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 770.864037] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 770.864037] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 770.864037] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 770.866065] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 770.866065] nova-conductor[52243]: Traceback (most recent call last): [ 770.866065] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 770.866065] nova-conductor[52243]: return func(*args, **kwargs) [ 770.866065] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 770.866065] nova-conductor[52243]: selections = self._select_destinations( [ 770.866065] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 770.866065] nova-conductor[52243]: selections = self._schedule( [ 770.866065] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 770.866065] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 770.866065] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 770.866065] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 770.866065] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 770.866065] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 770.868769] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] [instance: 9f290d28-a018-4a06-8c84-81463a83b0b2] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 770.884940] nova-conductor[52244]: DEBUG nova.network.neutron [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] [instance: 938a2016-8eaa-446a-b69c-3af59448d944] deallocate_for_instance() {{(pid=52244) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 770.891078] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 770.891078] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 770.891078] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 770.930892] nova-conductor[52244]: DEBUG nova.network.neutron [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] [instance: 938a2016-8eaa-446a-b69c-3af59448d944] Instance cache missing network info. {{(pid=52244) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 770.932142] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] [instance: 72a92098-562e-47bf-8dde-8b62b182d7bb] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1982, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1960, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1229, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1365, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1340, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 35682cb9-e9d7-4838-847e-d5bcc61f6753, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 72a92098-562e-47bf-8dde-8b62b182d7bb was re-scheduled: Binding failed for port 35682cb9-e9d7-4838-847e-d5bcc61f6753, please check neutron logs for more information.\n'] [ 770.932561] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 770.932789] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 72a92098-562e-47bf-8dde-8b62b182d7bb.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 72a92098-562e-47bf-8dde-8b62b182d7bb. [ 770.932997] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] [instance: 72a92098-562e-47bf-8dde-8b62b182d7bb] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 72a92098-562e-47bf-8dde-8b62b182d7bb. [ 770.941661] nova-conductor[52244]: DEBUG nova.network.neutron [None req-a286a9f0-510d-4024-931c-d2c960680507 tempest-ListServerFiltersTestJSON-278412246 tempest-ListServerFiltersTestJSON-278412246-project-member] [instance: 938a2016-8eaa-446a-b69c-3af59448d944] Updating instance_info_cache with network_info: [] {{(pid=52244) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 770.958680] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] [instance: e1c51f22-c186-4a2f-a2c9-827319b2d8e6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 770.959066] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 770.959282] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 770.959624] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 770.963209] nova-conductor[52243]: DEBUG nova.network.neutron [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] [instance: 72a92098-562e-47bf-8dde-8b62b182d7bb] deallocate_for_instance() {{(pid=52243) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1803}} [ 770.966195] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 770.966195] nova-conductor[52243]: Traceback (most recent call last): [ 770.966195] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 770.966195] nova-conductor[52243]: return func(*args, **kwargs) [ 770.966195] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 770.966195] nova-conductor[52243]: selections = self._select_destinations( [ 770.966195] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 770.966195] nova-conductor[52243]: selections = self._schedule( [ 770.966195] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 770.966195] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 770.966195] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 770.966195] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 770.966195] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 770.966195] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 770.967701] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-47dfadc9-479f-411b-b305-7f48f3905139 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] [instance: e1c51f22-c186-4a2f-a2c9-827319b2d8e6] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 771.001674] nova-conductor[52243]: DEBUG nova.network.neutron [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] [instance: 72a92098-562e-47bf-8dde-8b62b182d7bb] Instance cache missing network info. {{(pid=52243) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3323}} [ 771.004959] nova-conductor[52243]: DEBUG nova.network.neutron [None req-bbfcb3a2-6871-4da0-9d3f-41fc23dce976 tempest-AttachVolumeNegativeTest-1432714470 tempest-AttachVolumeNegativeTest-1432714470-project-member] [instance: 72a92098-562e-47bf-8dde-8b62b182d7bb] Updating instance_info_cache with network_info: [] {{(pid=52243) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager [None req-10d6eac8-35e9-4158-a24a-8956a32b0053 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 772.121887] nova-conductor[52243]: Traceback (most recent call last): [ 772.121887] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 772.121887] nova-conductor[52243]: return func(*args, **kwargs) [ 772.121887] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 772.121887] nova-conductor[52243]: selections = self._select_destinations( [ 772.121887] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 772.121887] nova-conductor[52243]: selections = self._schedule( [ 772.121887] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 772.121887] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 772.121887] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 772.121887] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 772.121887] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager [ 772.121887] nova-conductor[52243]: ERROR nova.conductor.manager [ 772.139022] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-10d6eac8-35e9-4158-a24a-8956a32b0053 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 772.139022] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-10d6eac8-35e9-4158-a24a-8956a32b0053 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 772.139022] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-10d6eac8-35e9-4158-a24a-8956a32b0053 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 772.200740] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-10d6eac8-35e9-4158-a24a-8956a32b0053 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: 47c85522-bb7b-4cf8-a959-9038d63eec79] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 772.201538] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-10d6eac8-35e9-4158-a24a-8956a32b0053 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 772.201784] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-10d6eac8-35e9-4158-a24a-8956a32b0053 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 772.202086] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-10d6eac8-35e9-4158-a24a-8956a32b0053 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 772.206283] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-10d6eac8-35e9-4158-a24a-8956a32b0053 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 772.206283] nova-conductor[52243]: Traceback (most recent call last): [ 772.206283] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 772.206283] nova-conductor[52243]: return func(*args, **kwargs) [ 772.206283] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 772.206283] nova-conductor[52243]: selections = self._select_destinations( [ 772.206283] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 772.206283] nova-conductor[52243]: selections = self._schedule( [ 772.206283] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 772.206283] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 772.206283] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 772.206283] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 772.206283] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 772.206283] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 772.206847] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-10d6eac8-35e9-4158-a24a-8956a32b0053 tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: 47c85522-bb7b-4cf8-a959-9038d63eec79] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager [None req-72720bf1-f30e-4c4d-9f15-6bb858a6d80b tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 776.306259] nova-conductor[52244]: Traceback (most recent call last): [ 776.306259] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 776.306259] nova-conductor[52244]: return func(*args, **kwargs) [ 776.306259] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 776.306259] nova-conductor[52244]: selections = self._select_destinations( [ 776.306259] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 776.306259] nova-conductor[52244]: selections = self._schedule( [ 776.306259] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 776.306259] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 776.306259] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 776.306259] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 776.306259] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager [ 776.306259] nova-conductor[52244]: ERROR nova.conductor.manager [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager [None req-5ae4decc-1ce4-4104-bfae-e90062f57462 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 776.314152] nova-conductor[52243]: Traceback (most recent call last): [ 776.314152] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 776.314152] nova-conductor[52243]: return func(*args, **kwargs) [ 776.314152] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 776.314152] nova-conductor[52243]: selections = self._select_destinations( [ 776.314152] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 776.314152] nova-conductor[52243]: selections = self._schedule( [ 776.314152] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 776.314152] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 776.314152] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 776.314152] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 776.314152] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager [ 776.314152] nova-conductor[52243]: ERROR nova.conductor.manager [ 776.316638] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-72720bf1-f30e-4c4d-9f15-6bb858a6d80b tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 776.316638] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-72720bf1-f30e-4c4d-9f15-6bb858a6d80b tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 776.316638] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-72720bf1-f30e-4c4d-9f15-6bb858a6d80b tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 776.320261] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5ae4decc-1ce4-4104-bfae-e90062f57462 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 776.320501] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5ae4decc-1ce4-4104-bfae-e90062f57462 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 776.320670] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5ae4decc-1ce4-4104-bfae-e90062f57462 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 776.359626] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-72720bf1-f30e-4c4d-9f15-6bb858a6d80b tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: 8510b3fe-38d8-4c49-aab0-76c709818fd7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 776.360170] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-5ae4decc-1ce4-4104-bfae-e90062f57462 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: 7962aa39-04ac-4865-b0e0-aa9119505d22] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 776.360481] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-72720bf1-f30e-4c4d-9f15-6bb858a6d80b tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 776.360701] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-72720bf1-f30e-4c4d-9f15-6bb858a6d80b tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 776.361129] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5ae4decc-1ce4-4104-bfae-e90062f57462 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 776.361210] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-72720bf1-f30e-4c4d-9f15-6bb858a6d80b tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 776.361244] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5ae4decc-1ce4-4104-bfae-e90062f57462 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 776.361244] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-5ae4decc-1ce4-4104-bfae-e90062f57462 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 776.363957] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-72720bf1-f30e-4c4d-9f15-6bb858a6d80b tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 776.363957] nova-conductor[52244]: Traceback (most recent call last): [ 776.363957] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 776.363957] nova-conductor[52244]: return func(*args, **kwargs) [ 776.363957] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 776.363957] nova-conductor[52244]: selections = self._select_destinations( [ 776.363957] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 776.363957] nova-conductor[52244]: selections = self._schedule( [ 776.363957] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 776.363957] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 776.363957] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 776.363957] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 776.363957] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 776.363957] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 776.364479] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-5ae4decc-1ce4-4104-bfae-e90062f57462 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 776.364479] nova-conductor[52243]: Traceback (most recent call last): [ 776.364479] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 776.364479] nova-conductor[52243]: return func(*args, **kwargs) [ 776.364479] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 776.364479] nova-conductor[52243]: selections = self._select_destinations( [ 776.364479] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 776.364479] nova-conductor[52243]: selections = self._schedule( [ 776.364479] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 776.364479] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 776.364479] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 776.364479] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 776.364479] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 776.364479] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 776.364843] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-72720bf1-f30e-4c4d-9f15-6bb858a6d80b tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: 8510b3fe-38d8-4c49-aab0-76c709818fd7] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 776.365236] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-5ae4decc-1ce4-4104-bfae-e90062f57462 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: 7962aa39-04ac-4865-b0e0-aa9119505d22] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager [None req-8a15b21a-b52e-412a-a9d3-2a4e31fe2c16 tempest-ServerAddressesTestJSON-145131219 tempest-ServerAddressesTestJSON-145131219-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 778.506715] nova-conductor[52244]: Traceback (most recent call last): [ 778.506715] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 778.506715] nova-conductor[52244]: return func(*args, **kwargs) [ 778.506715] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 778.506715] nova-conductor[52244]: selections = self._select_destinations( [ 778.506715] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 778.506715] nova-conductor[52244]: selections = self._schedule( [ 778.506715] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 778.506715] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 778.506715] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 778.506715] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 778.506715] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager [ 778.506715] nova-conductor[52244]: ERROR nova.conductor.manager [ 778.520123] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8a15b21a-b52e-412a-a9d3-2a4e31fe2c16 tempest-ServerAddressesTestJSON-145131219 tempest-ServerAddressesTestJSON-145131219-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 778.520769] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8a15b21a-b52e-412a-a9d3-2a4e31fe2c16 tempest-ServerAddressesTestJSON-145131219 tempest-ServerAddressesTestJSON-145131219-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 778.520769] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8a15b21a-b52e-412a-a9d3-2a4e31fe2c16 tempest-ServerAddressesTestJSON-145131219 tempest-ServerAddressesTestJSON-145131219-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 778.578244] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-8a15b21a-b52e-412a-a9d3-2a4e31fe2c16 tempest-ServerAddressesTestJSON-145131219 tempest-ServerAddressesTestJSON-145131219-project-member] [instance: 37925396-9a25-40a9-8eb0-9fb84dab1db6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 778.579059] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8a15b21a-b52e-412a-a9d3-2a4e31fe2c16 tempest-ServerAddressesTestJSON-145131219 tempest-ServerAddressesTestJSON-145131219-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 778.579287] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8a15b21a-b52e-412a-a9d3-2a4e31fe2c16 tempest-ServerAddressesTestJSON-145131219 tempest-ServerAddressesTestJSON-145131219-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 778.579461] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-8a15b21a-b52e-412a-a9d3-2a4e31fe2c16 tempest-ServerAddressesTestJSON-145131219 tempest-ServerAddressesTestJSON-145131219-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 778.585474] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-8a15b21a-b52e-412a-a9d3-2a4e31fe2c16 tempest-ServerAddressesTestJSON-145131219 tempest-ServerAddressesTestJSON-145131219-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 778.585474] nova-conductor[52244]: Traceback (most recent call last): [ 778.585474] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 778.585474] nova-conductor[52244]: return func(*args, **kwargs) [ 778.585474] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 778.585474] nova-conductor[52244]: selections = self._select_destinations( [ 778.585474] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 778.585474] nova-conductor[52244]: selections = self._schedule( [ 778.585474] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 778.585474] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 778.585474] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 778.585474] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 778.585474] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 778.585474] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 778.585474] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-8a15b21a-b52e-412a-a9d3-2a4e31fe2c16 tempest-ServerAddressesTestJSON-145131219 tempest-ServerAddressesTestJSON-145131219-project-member] [instance: 37925396-9a25-40a9-8eb0-9fb84dab1db6] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager [None req-6cb3a564-5218-4011-b742-fc28e350f35e tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 780.291301] nova-conductor[52243]: Traceback (most recent call last): [ 780.291301] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 780.291301] nova-conductor[52243]: return func(*args, **kwargs) [ 780.291301] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 780.291301] nova-conductor[52243]: selections = self._select_destinations( [ 780.291301] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 780.291301] nova-conductor[52243]: selections = self._schedule( [ 780.291301] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 780.291301] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 780.291301] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 780.291301] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 780.291301] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager [ 780.291301] nova-conductor[52243]: ERROR nova.conductor.manager [ 780.302576] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6cb3a564-5218-4011-b742-fc28e350f35e tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 780.305017] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6cb3a564-5218-4011-b742-fc28e350f35e tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 780.305017] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6cb3a564-5218-4011-b742-fc28e350f35e tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 780.365284] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-6cb3a564-5218-4011-b742-fc28e350f35e tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: 8b74fc20-68b5-4685-b079-472d8bae0f6f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 780.366183] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6cb3a564-5218-4011-b742-fc28e350f35e tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 780.366410] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6cb3a564-5218-4011-b742-fc28e350f35e tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 780.366623] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-6cb3a564-5218-4011-b742-fc28e350f35e tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 780.376031] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-6cb3a564-5218-4011-b742-fc28e350f35e tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 780.376031] nova-conductor[52243]: Traceback (most recent call last): [ 780.376031] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 780.376031] nova-conductor[52243]: return func(*args, **kwargs) [ 780.376031] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 780.376031] nova-conductor[52243]: selections = self._select_destinations( [ 780.376031] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 780.376031] nova-conductor[52243]: selections = self._schedule( [ 780.376031] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 780.376031] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 780.376031] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 780.376031] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 780.376031] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 780.376031] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 780.376566] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-6cb3a564-5218-4011-b742-fc28e350f35e tempest-ImagesTestJSON-1854810537 tempest-ImagesTestJSON-1854810537-project-member] [instance: 8b74fc20-68b5-4685-b079-472d8bae0f6f] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager [None req-c0eaa05a-193b-4839-9d73-31ef689e4d13 tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 780.744321] nova-conductor[52244]: Traceback (most recent call last): [ 780.744321] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 780.744321] nova-conductor[52244]: return func(*args, **kwargs) [ 780.744321] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 780.744321] nova-conductor[52244]: selections = self._select_destinations( [ 780.744321] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 780.744321] nova-conductor[52244]: selections = self._schedule( [ 780.744321] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 780.744321] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 780.744321] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 780.744321] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 780.744321] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager [ 780.744321] nova-conductor[52244]: ERROR nova.conductor.manager [ 780.752508] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-c0eaa05a-193b-4839-9d73-31ef689e4d13 tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 780.752744] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-c0eaa05a-193b-4839-9d73-31ef689e4d13 tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 780.752916] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-c0eaa05a-193b-4839-9d73-31ef689e4d13 tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 780.807887] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-c0eaa05a-193b-4839-9d73-31ef689e4d13 tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] [instance: 50e91d7a-911b-4b9c-9f69-6180335366c0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 780.808740] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-c0eaa05a-193b-4839-9d73-31ef689e4d13 tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 780.808939] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-c0eaa05a-193b-4839-9d73-31ef689e4d13 tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 780.809120] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-c0eaa05a-193b-4839-9d73-31ef689e4d13 tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 780.816072] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-c0eaa05a-193b-4839-9d73-31ef689e4d13 tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 780.816072] nova-conductor[52244]: Traceback (most recent call last): [ 780.816072] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 780.816072] nova-conductor[52244]: return func(*args, **kwargs) [ 780.816072] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 780.816072] nova-conductor[52244]: selections = self._select_destinations( [ 780.816072] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 780.816072] nova-conductor[52244]: selections = self._schedule( [ 780.816072] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 780.816072] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 780.816072] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 780.816072] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 780.816072] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 780.816072] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 780.816072] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-c0eaa05a-193b-4839-9d73-31ef689e4d13 tempest-SecurityGroupsTestJSON-536329627 tempest-SecurityGroupsTestJSON-536329627-project-member] [instance: 50e91d7a-911b-4b9c-9f69-6180335366c0] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager [None req-41dd8cee-006e-49de-860a-6072d876bbd7 tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 780.924540] nova-conductor[52243]: Traceback (most recent call last): [ 780.924540] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 780.924540] nova-conductor[52243]: return func(*args, **kwargs) [ 780.924540] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 780.924540] nova-conductor[52243]: selections = self._select_destinations( [ 780.924540] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 780.924540] nova-conductor[52243]: selections = self._schedule( [ 780.924540] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 780.924540] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 780.924540] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 780.924540] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 780.924540] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager [ 780.924540] nova-conductor[52243]: ERROR nova.conductor.manager [ 780.936511] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-41dd8cee-006e-49de-860a-6072d876bbd7 tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 780.938999] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-41dd8cee-006e-49de-860a-6072d876bbd7 tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 780.938999] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-41dd8cee-006e-49de-860a-6072d876bbd7 tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 780.987905] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-41dd8cee-006e-49de-860a-6072d876bbd7 tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] [instance: 46f9cfe6-f752-4aae-aa30-b2cbbd42b4df] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 780.988685] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-41dd8cee-006e-49de-860a-6072d876bbd7 tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 780.988883] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-41dd8cee-006e-49de-860a-6072d876bbd7 tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 780.988999] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-41dd8cee-006e-49de-860a-6072d876bbd7 tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 780.993586] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-41dd8cee-006e-49de-860a-6072d876bbd7 tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 780.993586] nova-conductor[52243]: Traceback (most recent call last): [ 780.993586] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 780.993586] nova-conductor[52243]: return func(*args, **kwargs) [ 780.993586] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 780.993586] nova-conductor[52243]: selections = self._select_destinations( [ 780.993586] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 780.993586] nova-conductor[52243]: selections = self._schedule( [ 780.993586] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 780.993586] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 780.993586] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 780.993586] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 780.993586] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 780.993586] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 780.994181] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-41dd8cee-006e-49de-860a-6072d876bbd7 tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] [instance: 46f9cfe6-f752-4aae-aa30-b2cbbd42b4df] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager [None req-66f7dbf4-8e9d-440c-bf9c-c8932d056d2c tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 781.847322] nova-conductor[52244]: Traceback (most recent call last): [ 781.847322] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 781.847322] nova-conductor[52244]: return func(*args, **kwargs) [ 781.847322] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 781.847322] nova-conductor[52244]: selections = self._select_destinations( [ 781.847322] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 781.847322] nova-conductor[52244]: selections = self._schedule( [ 781.847322] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 781.847322] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 781.847322] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 781.847322] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 781.847322] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager [ 781.847322] nova-conductor[52244]: ERROR nova.conductor.manager [ 781.857075] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-66f7dbf4-8e9d-440c-bf9c-c8932d056d2c tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 781.859938] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-66f7dbf4-8e9d-440c-bf9c-c8932d056d2c tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 781.859938] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-66f7dbf4-8e9d-440c-bf9c-c8932d056d2c tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 781.919199] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-66f7dbf4-8e9d-440c-bf9c-c8932d056d2c tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] [instance: 52d42714-e52c-48b3-ae4a-49715fbff016] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 781.919908] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-66f7dbf4-8e9d-440c-bf9c-c8932d056d2c tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 781.920139] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-66f7dbf4-8e9d-440c-bf9c-c8932d056d2c tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 781.920313] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-66f7dbf4-8e9d-440c-bf9c-c8932d056d2c tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 781.924173] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-66f7dbf4-8e9d-440c-bf9c-c8932d056d2c tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 781.924173] nova-conductor[52244]: Traceback (most recent call last): [ 781.924173] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 781.924173] nova-conductor[52244]: return func(*args, **kwargs) [ 781.924173] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 781.924173] nova-conductor[52244]: selections = self._select_destinations( [ 781.924173] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 781.924173] nova-conductor[52244]: selections = self._schedule( [ 781.924173] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 781.924173] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 781.924173] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 781.924173] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 781.924173] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 781.924173] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 781.924694] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-66f7dbf4-8e9d-440c-bf9c-c8932d056d2c tempest-ListImageFiltersTestJSON-614978481 tempest-ListImageFiltersTestJSON-614978481-project-member] [instance: 52d42714-e52c-48b3-ae4a-49715fbff016] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 782.562275] nova-conductor[52243]: Traceback (most recent call last): [ 782.562275] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 782.562275] nova-conductor[52243]: return func(*args, **kwargs) [ 782.562275] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 782.562275] nova-conductor[52243]: selections = self._select_destinations( [ 782.562275] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 782.562275] nova-conductor[52243]: selections = self._schedule( [ 782.562275] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 782.562275] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 782.562275] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 782.562275] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 782.562275] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager [ 782.562275] nova-conductor[52243]: ERROR nova.conductor.manager [ 782.577267] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 782.578703] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 782.578703] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 782.630230] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] [instance: 0b0f0761-8b9f-4975-a7ab-bf4bd2686d04] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 782.631534] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 782.631867] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 782.632164] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 782.636446] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 782.636446] nova-conductor[52243]: Traceback (most recent call last): [ 782.636446] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 782.636446] nova-conductor[52243]: return func(*args, **kwargs) [ 782.636446] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 782.636446] nova-conductor[52243]: selections = self._select_destinations( [ 782.636446] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 782.636446] nova-conductor[52243]: selections = self._schedule( [ 782.636446] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 782.636446] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 782.636446] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 782.636446] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 782.636446] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 782.636446] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 782.639058] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] [instance: 0b0f0761-8b9f-4975-a7ab-bf4bd2686d04] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 782.669426] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 782.669733] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 782.669919] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager [None req-851ee1f7-492b-49b0-9601-8487b964d883 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 782.697758] nova-conductor[52244]: Traceback (most recent call last): [ 782.697758] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 782.697758] nova-conductor[52244]: return func(*args, **kwargs) [ 782.697758] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 782.697758] nova-conductor[52244]: selections = self._select_destinations( [ 782.697758] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 782.697758] nova-conductor[52244]: selections = self._schedule( [ 782.697758] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 782.697758] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 782.697758] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 782.697758] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 782.697758] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager [ 782.697758] nova-conductor[52244]: ERROR nova.conductor.manager [ 782.711902] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-851ee1f7-492b-49b0-9601-8487b964d883 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 782.711902] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-851ee1f7-492b-49b0-9601-8487b964d883 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 782.713033] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-851ee1f7-492b-49b0-9601-8487b964d883 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 782.713985] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] [instance: 0b203a37-91eb-47b3-ad22-99db57d9e7ed] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 782.714439] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 782.714654] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 782.714820] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 782.718394] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 782.718394] nova-conductor[52243]: Traceback (most recent call last): [ 782.718394] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 782.718394] nova-conductor[52243]: return func(*args, **kwargs) [ 782.718394] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 782.718394] nova-conductor[52243]: selections = self._select_destinations( [ 782.718394] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 782.718394] nova-conductor[52243]: selections = self._schedule( [ 782.718394] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 782.718394] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 782.718394] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 782.718394] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 782.718394] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 782.718394] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 782.719021] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-dfb02a0e-837f-4400-ae82-b2f12f29f6d1 tempest-MultipleCreateTestJSON-1191137847 tempest-MultipleCreateTestJSON-1191137847-project-member] [instance: 0b203a37-91eb-47b3-ad22-99db57d9e7ed] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 782.762922] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-851ee1f7-492b-49b0-9601-8487b964d883 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: 94758992-1d4a-492d-b4d7-30443aa3bbe7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 782.764062] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-851ee1f7-492b-49b0-9601-8487b964d883 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 782.764062] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-851ee1f7-492b-49b0-9601-8487b964d883 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 782.764062] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-851ee1f7-492b-49b0-9601-8487b964d883 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 782.767982] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-851ee1f7-492b-49b0-9601-8487b964d883 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 782.767982] nova-conductor[52244]: Traceback (most recent call last): [ 782.767982] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 782.767982] nova-conductor[52244]: return func(*args, **kwargs) [ 782.767982] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 782.767982] nova-conductor[52244]: selections = self._select_destinations( [ 782.767982] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 782.767982] nova-conductor[52244]: selections = self._schedule( [ 782.767982] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 782.767982] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 782.767982] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 782.767982] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 782.767982] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 782.767982] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 782.768569] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-851ee1f7-492b-49b0-9601-8487b964d883 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: 94758992-1d4a-492d-b4d7-30443aa3bbe7] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager [None req-b3e235fe-7eac-4674-91e1-f66d3ea51a37 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 783.310591] nova-conductor[52243]: Traceback (most recent call last): [ 783.310591] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 783.310591] nova-conductor[52243]: return func(*args, **kwargs) [ 783.310591] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 783.310591] nova-conductor[52243]: selections = self._select_destinations( [ 783.310591] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 783.310591] nova-conductor[52243]: selections = self._schedule( [ 783.310591] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 783.310591] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 783.310591] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 783.310591] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 783.310591] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager [ 783.310591] nova-conductor[52243]: ERROR nova.conductor.manager [ 783.318015] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b3e235fe-7eac-4674-91e1-f66d3ea51a37 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 783.318491] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b3e235fe-7eac-4674-91e1-f66d3ea51a37 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 783.318491] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b3e235fe-7eac-4674-91e1-f66d3ea51a37 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 783.362697] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-b3e235fe-7eac-4674-91e1-f66d3ea51a37 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: 56f16d5e-8949-4a36-8bc1-9981e1d190ab] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 783.363746] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b3e235fe-7eac-4674-91e1-f66d3ea51a37 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 783.363963] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b3e235fe-7eac-4674-91e1-f66d3ea51a37 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 783.364159] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-b3e235fe-7eac-4674-91e1-f66d3ea51a37 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 783.367504] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-b3e235fe-7eac-4674-91e1-f66d3ea51a37 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 783.367504] nova-conductor[52243]: Traceback (most recent call last): [ 783.367504] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 783.367504] nova-conductor[52243]: return func(*args, **kwargs) [ 783.367504] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 783.367504] nova-conductor[52243]: selections = self._select_destinations( [ 783.367504] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 783.367504] nova-conductor[52243]: selections = self._schedule( [ 783.367504] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 783.367504] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 783.367504] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 783.367504] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 783.367504] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 783.367504] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 783.368087] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-b3e235fe-7eac-4674-91e1-f66d3ea51a37 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: 56f16d5e-8949-4a36-8bc1-9981e1d190ab] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager [None req-a5377b21-394c-4888-9ee8-48923c77f4db tempest-ServerActionsTestOtherA-1173896838 tempest-ServerActionsTestOtherA-1173896838-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 784.987134] nova-conductor[52244]: Traceback (most recent call last): [ 784.987134] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 784.987134] nova-conductor[52244]: return func(*args, **kwargs) [ 784.987134] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 784.987134] nova-conductor[52244]: selections = self._select_destinations( [ 784.987134] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 784.987134] nova-conductor[52244]: selections = self._schedule( [ 784.987134] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 784.987134] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 784.987134] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 784.987134] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 784.987134] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager [ 784.987134] nova-conductor[52244]: ERROR nova.conductor.manager [ 784.995654] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a5377b21-394c-4888-9ee8-48923c77f4db tempest-ServerActionsTestOtherA-1173896838 tempest-ServerActionsTestOtherA-1173896838-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 784.995654] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a5377b21-394c-4888-9ee8-48923c77f4db tempest-ServerActionsTestOtherA-1173896838 tempest-ServerActionsTestOtherA-1173896838-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 784.995654] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a5377b21-394c-4888-9ee8-48923c77f4db tempest-ServerActionsTestOtherA-1173896838 tempest-ServerActionsTestOtherA-1173896838-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 785.049781] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-a5377b21-394c-4888-9ee8-48923c77f4db tempest-ServerActionsTestOtherA-1173896838 tempest-ServerActionsTestOtherA-1173896838-project-member] [instance: 62b668da-d57c-405a-8c37-63a15f2a506e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 785.049781] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a5377b21-394c-4888-9ee8-48923c77f4db tempest-ServerActionsTestOtherA-1173896838 tempest-ServerActionsTestOtherA-1173896838-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 785.049995] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a5377b21-394c-4888-9ee8-48923c77f4db tempest-ServerActionsTestOtherA-1173896838 tempest-ServerActionsTestOtherA-1173896838-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 785.050089] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-a5377b21-394c-4888-9ee8-48923c77f4db tempest-ServerActionsTestOtherA-1173896838 tempest-ServerActionsTestOtherA-1173896838-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 785.053680] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-a5377b21-394c-4888-9ee8-48923c77f4db tempest-ServerActionsTestOtherA-1173896838 tempest-ServerActionsTestOtherA-1173896838-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 785.053680] nova-conductor[52244]: Traceback (most recent call last): [ 785.053680] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 785.053680] nova-conductor[52244]: return func(*args, **kwargs) [ 785.053680] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 785.053680] nova-conductor[52244]: selections = self._select_destinations( [ 785.053680] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 785.053680] nova-conductor[52244]: selections = self._schedule( [ 785.053680] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 785.053680] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 785.053680] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 785.053680] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 785.053680] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 785.053680] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 785.054705] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-a5377b21-394c-4888-9ee8-48923c77f4db tempest-ServerActionsTestOtherA-1173896838 tempest-ServerActionsTestOtherA-1173896838-project-member] [instance: 62b668da-d57c-405a-8c37-63a15f2a506e] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager [None req-a0af0ff3-ca4f-4720-8553-a6476e97d266 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 787.269448] nova-conductor[52243]: Traceback (most recent call last): [ 787.269448] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 787.269448] nova-conductor[52243]: return func(*args, **kwargs) [ 787.269448] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 787.269448] nova-conductor[52243]: selections = self._select_destinations( [ 787.269448] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 787.269448] nova-conductor[52243]: selections = self._schedule( [ 787.269448] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 787.269448] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 787.269448] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 787.269448] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 787.269448] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager [ 787.269448] nova-conductor[52243]: ERROR nova.conductor.manager [ 787.284963] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a0af0ff3-ca4f-4720-8553-a6476e97d266 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 787.285450] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a0af0ff3-ca4f-4720-8553-a6476e97d266 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 787.285686] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a0af0ff3-ca4f-4720-8553-a6476e97d266 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 787.353413] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-a0af0ff3-ca4f-4720-8553-a6476e97d266 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: a9f5cfbc-2394-4eaf-bba4-b6a17cf29dcd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 787.354064] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a0af0ff3-ca4f-4720-8553-a6476e97d266 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 787.354277] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a0af0ff3-ca4f-4720-8553-a6476e97d266 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 787.354458] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-a0af0ff3-ca4f-4720-8553-a6476e97d266 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 787.357764] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-a0af0ff3-ca4f-4720-8553-a6476e97d266 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 787.357764] nova-conductor[52243]: Traceback (most recent call last): [ 787.357764] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 787.357764] nova-conductor[52243]: return func(*args, **kwargs) [ 787.357764] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 787.357764] nova-conductor[52243]: selections = self._select_destinations( [ 787.357764] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 787.357764] nova-conductor[52243]: selections = self._schedule( [ 787.357764] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 787.357764] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 787.357764] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 787.357764] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 787.357764] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 787.357764] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 787.359353] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-a0af0ff3-ca4f-4720-8553-a6476e97d266 tempest-DeleteServersTestJSON-1071653592 tempest-DeleteServersTestJSON-1071653592-project-member] [instance: a9f5cfbc-2394-4eaf-bba4-b6a17cf29dcd] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager [None req-0f1aaa9b-a0fa-43fe-baac-3db3a69fd88a tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 787.380998] nova-conductor[52244]: Traceback (most recent call last): [ 787.380998] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 787.380998] nova-conductor[52244]: return func(*args, **kwargs) [ 787.380998] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 787.380998] nova-conductor[52244]: selections = self._select_destinations( [ 787.380998] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 787.380998] nova-conductor[52244]: selections = self._schedule( [ 787.380998] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 787.380998] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 787.380998] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 787.380998] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 787.380998] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager [ 787.380998] nova-conductor[52244]: ERROR nova.conductor.manager [ 787.390877] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-0f1aaa9b-a0fa-43fe-baac-3db3a69fd88a tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 787.391129] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-0f1aaa9b-a0fa-43fe-baac-3db3a69fd88a tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 787.391297] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-0f1aaa9b-a0fa-43fe-baac-3db3a69fd88a tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 787.447078] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-0f1aaa9b-a0fa-43fe-baac-3db3a69fd88a tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: 503ed475-7585-4c26-9984-6eaa33bb97ca] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 787.448084] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-0f1aaa9b-a0fa-43fe-baac-3db3a69fd88a tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 787.450861] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-0f1aaa9b-a0fa-43fe-baac-3db3a69fd88a tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 787.450861] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-0f1aaa9b-a0fa-43fe-baac-3db3a69fd88a tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 787.459023] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-0f1aaa9b-a0fa-43fe-baac-3db3a69fd88a tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 787.459023] nova-conductor[52244]: Traceback (most recent call last): [ 787.459023] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 787.459023] nova-conductor[52244]: return func(*args, **kwargs) [ 787.459023] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 787.459023] nova-conductor[52244]: selections = self._select_destinations( [ 787.459023] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 787.459023] nova-conductor[52244]: selections = self._schedule( [ 787.459023] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 787.459023] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 787.459023] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 787.459023] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 787.459023] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 787.459023] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 787.459023] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-0f1aaa9b-a0fa-43fe-baac-3db3a69fd88a tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: 503ed475-7585-4c26-9984-6eaa33bb97ca] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager [None req-bf7856c1-a3b0-47da-b203-490070191afc tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 788.830135] nova-conductor[52243]: Traceback (most recent call last): [ 788.830135] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 788.830135] nova-conductor[52243]: return func(*args, **kwargs) [ 788.830135] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 788.830135] nova-conductor[52243]: selections = self._select_destinations( [ 788.830135] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 788.830135] nova-conductor[52243]: selections = self._schedule( [ 788.830135] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 788.830135] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 788.830135] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 788.830135] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 788.830135] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager [ 788.830135] nova-conductor[52243]: ERROR nova.conductor.manager [ 788.837906] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bf7856c1-a3b0-47da-b203-490070191afc tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 788.838124] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bf7856c1-a3b0-47da-b203-490070191afc tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 788.838822] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bf7856c1-a3b0-47da-b203-490070191afc tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 788.889164] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-bf7856c1-a3b0-47da-b203-490070191afc tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] [instance: 8d0e993e-5738-48d9-a97c-757fe32e82be] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 788.889661] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bf7856c1-a3b0-47da-b203-490070191afc tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 788.889867] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bf7856c1-a3b0-47da-b203-490070191afc tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 788.890049] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-bf7856c1-a3b0-47da-b203-490070191afc tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 788.893376] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-bf7856c1-a3b0-47da-b203-490070191afc tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 788.893376] nova-conductor[52243]: Traceback (most recent call last): [ 788.893376] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 788.893376] nova-conductor[52243]: return func(*args, **kwargs) [ 788.893376] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 788.893376] nova-conductor[52243]: selections = self._select_destinations( [ 788.893376] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 788.893376] nova-conductor[52243]: selections = self._schedule( [ 788.893376] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 788.893376] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 788.893376] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 788.893376] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 788.893376] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 788.893376] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 788.893905] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-bf7856c1-a3b0-47da-b203-490070191afc tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] [instance: 8d0e993e-5738-48d9-a97c-757fe32e82be] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager [None req-41bd279b-cee4-45a7-a0f6-9866a25d9a5a tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 789.916877] nova-conductor[52244]: Traceback (most recent call last): [ 789.916877] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 789.916877] nova-conductor[52244]: return func(*args, **kwargs) [ 789.916877] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 789.916877] nova-conductor[52244]: selections = self._select_destinations( [ 789.916877] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 789.916877] nova-conductor[52244]: selections = self._schedule( [ 789.916877] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 789.916877] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 789.916877] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 789.916877] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 789.916877] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager [ 789.916877] nova-conductor[52244]: ERROR nova.conductor.manager [ 789.930131] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-41bd279b-cee4-45a7-a0f6-9866a25d9a5a tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 789.930131] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-41bd279b-cee4-45a7-a0f6-9866a25d9a5a tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 789.930131] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-41bd279b-cee4-45a7-a0f6-9866a25d9a5a tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager [None req-d686fee6-a757-4ac8-b02a-df6e161f8a13 tempest-ServersNegativeTestJSON-1264316097 tempest-ServersNegativeTestJSON-1264316097-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 789.943769] nova-conductor[52243]: Traceback (most recent call last): [ 789.943769] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 789.943769] nova-conductor[52243]: return func(*args, **kwargs) [ 789.943769] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 789.943769] nova-conductor[52243]: selections = self._select_destinations( [ 789.943769] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 789.943769] nova-conductor[52243]: selections = self._schedule( [ 789.943769] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 789.943769] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 789.943769] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 789.943769] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 789.943769] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager [ 789.943769] nova-conductor[52243]: ERROR nova.conductor.manager [ 789.953581] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-d686fee6-a757-4ac8-b02a-df6e161f8a13 tempest-ServersNegativeTestJSON-1264316097 tempest-ServersNegativeTestJSON-1264316097-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 789.953581] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-d686fee6-a757-4ac8-b02a-df6e161f8a13 tempest-ServersNegativeTestJSON-1264316097 tempest-ServersNegativeTestJSON-1264316097-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 789.953581] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-d686fee6-a757-4ac8-b02a-df6e161f8a13 tempest-ServersNegativeTestJSON-1264316097 tempest-ServersNegativeTestJSON-1264316097-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 789.984644] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-41bd279b-cee4-45a7-a0f6-9866a25d9a5a tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] [instance: a8cedd9a-605a-4e00-9f52-8e25b600cc92] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 789.985643] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-41bd279b-cee4-45a7-a0f6-9866a25d9a5a tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 789.987419] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-41bd279b-cee4-45a7-a0f6-9866a25d9a5a tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 789.987419] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-41bd279b-cee4-45a7-a0f6-9866a25d9a5a tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 789.990524] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-41bd279b-cee4-45a7-a0f6-9866a25d9a5a tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 789.990524] nova-conductor[52244]: Traceback (most recent call last): [ 789.990524] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 789.990524] nova-conductor[52244]: return func(*args, **kwargs) [ 789.990524] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 789.990524] nova-conductor[52244]: selections = self._select_destinations( [ 789.990524] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 789.990524] nova-conductor[52244]: selections = self._schedule( [ 789.990524] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 789.990524] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 789.990524] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 789.990524] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 789.990524] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 789.990524] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 789.991335] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-41bd279b-cee4-45a7-a0f6-9866a25d9a5a tempest-ServerRescueNegativeTestJSON-188112280 tempest-ServerRescueNegativeTestJSON-188112280-project-member] [instance: a8cedd9a-605a-4e00-9f52-8e25b600cc92] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 790.006669] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-d686fee6-a757-4ac8-b02a-df6e161f8a13 tempest-ServersNegativeTestJSON-1264316097 tempest-ServersNegativeTestJSON-1264316097-project-member] [instance: 01edd188-836b-4aac-8a4d-321153d00202] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 790.006669] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-d686fee6-a757-4ac8-b02a-df6e161f8a13 tempest-ServersNegativeTestJSON-1264316097 tempest-ServersNegativeTestJSON-1264316097-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 790.006894] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-d686fee6-a757-4ac8-b02a-df6e161f8a13 tempest-ServersNegativeTestJSON-1264316097 tempest-ServersNegativeTestJSON-1264316097-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 790.007082] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-d686fee6-a757-4ac8-b02a-df6e161f8a13 tempest-ServersNegativeTestJSON-1264316097 tempest-ServersNegativeTestJSON-1264316097-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 790.011765] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-d686fee6-a757-4ac8-b02a-df6e161f8a13 tempest-ServersNegativeTestJSON-1264316097 tempest-ServersNegativeTestJSON-1264316097-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 790.011765] nova-conductor[52243]: Traceback (most recent call last): [ 790.011765] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 790.011765] nova-conductor[52243]: return func(*args, **kwargs) [ 790.011765] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 790.011765] nova-conductor[52243]: selections = self._select_destinations( [ 790.011765] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 790.011765] nova-conductor[52243]: selections = self._schedule( [ 790.011765] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 790.011765] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 790.011765] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 790.011765] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 790.011765] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 790.011765] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 790.013452] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-d686fee6-a757-4ac8-b02a-df6e161f8a13 tempest-ServersNegativeTestJSON-1264316097 tempest-ServersNegativeTestJSON-1264316097-project-member] [instance: 01edd188-836b-4aac-8a4d-321153d00202] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager [None req-68fe73da-00c3-4814-93d7-17bcef934acb tempest-ServerAddressesNegativeTestJSON-841234166 tempest-ServerAddressesNegativeTestJSON-841234166-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 790.567993] nova-conductor[52244]: Traceback (most recent call last): [ 790.567993] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 790.567993] nova-conductor[52244]: return func(*args, **kwargs) [ 790.567993] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 790.567993] nova-conductor[52244]: selections = self._select_destinations( [ 790.567993] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 790.567993] nova-conductor[52244]: selections = self._schedule( [ 790.567993] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 790.567993] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 790.567993] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 790.567993] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 790.567993] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager [ 790.567993] nova-conductor[52244]: ERROR nova.conductor.manager [ 790.574841] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-68fe73da-00c3-4814-93d7-17bcef934acb tempest-ServerAddressesNegativeTestJSON-841234166 tempest-ServerAddressesNegativeTestJSON-841234166-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 790.575110] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-68fe73da-00c3-4814-93d7-17bcef934acb tempest-ServerAddressesNegativeTestJSON-841234166 tempest-ServerAddressesNegativeTestJSON-841234166-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 790.575279] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-68fe73da-00c3-4814-93d7-17bcef934acb tempest-ServerAddressesNegativeTestJSON-841234166 tempest-ServerAddressesNegativeTestJSON-841234166-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 790.629825] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-68fe73da-00c3-4814-93d7-17bcef934acb tempest-ServerAddressesNegativeTestJSON-841234166 tempest-ServerAddressesNegativeTestJSON-841234166-project-member] [instance: 652f04cb-87ca-42a6-9331-df1064070e7f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 790.630924] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-68fe73da-00c3-4814-93d7-17bcef934acb tempest-ServerAddressesNegativeTestJSON-841234166 tempest-ServerAddressesNegativeTestJSON-841234166-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 790.631242] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-68fe73da-00c3-4814-93d7-17bcef934acb tempest-ServerAddressesNegativeTestJSON-841234166 tempest-ServerAddressesNegativeTestJSON-841234166-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 790.631477] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-68fe73da-00c3-4814-93d7-17bcef934acb tempest-ServerAddressesNegativeTestJSON-841234166 tempest-ServerAddressesNegativeTestJSON-841234166-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 790.636095] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-68fe73da-00c3-4814-93d7-17bcef934acb tempest-ServerAddressesNegativeTestJSON-841234166 tempest-ServerAddressesNegativeTestJSON-841234166-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 790.636095] nova-conductor[52244]: Traceback (most recent call last): [ 790.636095] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 790.636095] nova-conductor[52244]: return func(*args, **kwargs) [ 790.636095] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 790.636095] nova-conductor[52244]: selections = self._select_destinations( [ 790.636095] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 790.636095] nova-conductor[52244]: selections = self._schedule( [ 790.636095] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 790.636095] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 790.636095] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 790.636095] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 790.636095] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 790.636095] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 790.637547] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-68fe73da-00c3-4814-93d7-17bcef934acb tempest-ServerAddressesNegativeTestJSON-841234166 tempest-ServerAddressesNegativeTestJSON-841234166-project-member] [instance: 652f04cb-87ca-42a6-9331-df1064070e7f] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager [None req-2bc2b86d-cf55-4390-afb8-05199aef22e1 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 792.036581] nova-conductor[52243]: Traceback (most recent call last): [ 792.036581] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 792.036581] nova-conductor[52243]: return func(*args, **kwargs) [ 792.036581] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 792.036581] nova-conductor[52243]: selections = self._select_destinations( [ 792.036581] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 792.036581] nova-conductor[52243]: selections = self._schedule( [ 792.036581] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 792.036581] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 792.036581] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 792.036581] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 792.036581] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager [ 792.036581] nova-conductor[52243]: ERROR nova.conductor.manager [ 792.043256] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-2bc2b86d-cf55-4390-afb8-05199aef22e1 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 792.043382] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-2bc2b86d-cf55-4390-afb8-05199aef22e1 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 792.043950] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-2bc2b86d-cf55-4390-afb8-05199aef22e1 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 792.094112] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-2bc2b86d-cf55-4390-afb8-05199aef22e1 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: ffb89221-a83d-49e9-be1f-d1b7cdb414dd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 792.094689] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-2bc2b86d-cf55-4390-afb8-05199aef22e1 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 792.094914] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-2bc2b86d-cf55-4390-afb8-05199aef22e1 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 792.095577] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-2bc2b86d-cf55-4390-afb8-05199aef22e1 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 792.099519] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-2bc2b86d-cf55-4390-afb8-05199aef22e1 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 792.099519] nova-conductor[52243]: Traceback (most recent call last): [ 792.099519] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 792.099519] nova-conductor[52243]: return func(*args, **kwargs) [ 792.099519] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 792.099519] nova-conductor[52243]: selections = self._select_destinations( [ 792.099519] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 792.099519] nova-conductor[52243]: selections = self._schedule( [ 792.099519] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 792.099519] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 792.099519] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 792.099519] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 792.099519] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 792.099519] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 792.099519] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-2bc2b86d-cf55-4390-afb8-05199aef22e1 tempest-ServersTestJSON-1247949540 tempest-ServersTestJSON-1247949540-project-member] [instance: ffb89221-a83d-49e9-be1f-d1b7cdb414dd] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager [None req-72fb459b-7e0c-4a4f-bc86-49277751ccb6 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 796.255462] nova-conductor[52243]: Traceback (most recent call last): [ 796.255462] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 796.255462] nova-conductor[52243]: return func(*args, **kwargs) [ 796.255462] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 796.255462] nova-conductor[52243]: selections = self._select_destinations( [ 796.255462] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 796.255462] nova-conductor[52243]: selections = self._schedule( [ 796.255462] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 796.255462] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 796.255462] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 796.255462] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 796.255462] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager [ 796.255462] nova-conductor[52243]: ERROR nova.conductor.manager [ 796.267274] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-72fb459b-7e0c-4a4f-bc86-49277751ccb6 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 796.270634] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-72fb459b-7e0c-4a4f-bc86-49277751ccb6 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 796.270634] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-72fb459b-7e0c-4a4f-bc86-49277751ccb6 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 796.316538] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-72fb459b-7e0c-4a4f-bc86-49277751ccb6 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] [instance: 631b9fc0-bd05-4743-840c-7e6aad9207a8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 796.317308] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-72fb459b-7e0c-4a4f-bc86-49277751ccb6 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 796.317515] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-72fb459b-7e0c-4a4f-bc86-49277751ccb6 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 796.317680] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-72fb459b-7e0c-4a4f-bc86-49277751ccb6 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 796.321281] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-72fb459b-7e0c-4a4f-bc86-49277751ccb6 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 796.321281] nova-conductor[52243]: Traceback (most recent call last): [ 796.321281] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 796.321281] nova-conductor[52243]: return func(*args, **kwargs) [ 796.321281] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 796.321281] nova-conductor[52243]: selections = self._select_destinations( [ 796.321281] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 796.321281] nova-conductor[52243]: selections = self._schedule( [ 796.321281] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 796.321281] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 796.321281] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 796.321281] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 796.321281] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 796.321281] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 796.321979] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-72fb459b-7e0c-4a4f-bc86-49277751ccb6 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] [instance: 631b9fc0-bd05-4743-840c-7e6aad9207a8] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 798.205356] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] [instance: 78ed17da-e8e8-4872-b1bf-95c4e77de8e6] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 78ed17da-e8e8-4872-b1bf-95c4e77de8e6 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 798.205356] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 798.205356] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 78ed17da-e8e8-4872-b1bf-95c4e77de8e6.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 78ed17da-e8e8-4872-b1bf-95c4e77de8e6. [ 798.205356] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-8b302e70-ca17-499e-9ee1-9a7ccf961b0c tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] [instance: 78ed17da-e8e8-4872-b1bf-95c4e77de8e6] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 78ed17da-e8e8-4872-b1bf-95c4e77de8e6. [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager [None req-80cfb065-a581-4f40-9311-a3271ce83134 tempest-ServerShowV254Test-185728470 tempest-ServerShowV254Test-185728470-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 798.580072] nova-conductor[52243]: Traceback (most recent call last): [ 798.580072] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 798.580072] nova-conductor[52243]: return func(*args, **kwargs) [ 798.580072] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 798.580072] nova-conductor[52243]: selections = self._select_destinations( [ 798.580072] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 798.580072] nova-conductor[52243]: selections = self._schedule( [ 798.580072] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 798.580072] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 798.580072] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 798.580072] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 798.580072] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager [ 798.580072] nova-conductor[52243]: ERROR nova.conductor.manager [ 798.595750] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-80cfb065-a581-4f40-9311-a3271ce83134 tempest-ServerShowV254Test-185728470 tempest-ServerShowV254Test-185728470-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 798.597209] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-80cfb065-a581-4f40-9311-a3271ce83134 tempest-ServerShowV254Test-185728470 tempest-ServerShowV254Test-185728470-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 798.597209] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-80cfb065-a581-4f40-9311-a3271ce83134 tempest-ServerShowV254Test-185728470 tempest-ServerShowV254Test-185728470-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 798.657032] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-80cfb065-a581-4f40-9311-a3271ce83134 tempest-ServerShowV254Test-185728470 tempest-ServerShowV254Test-185728470-project-member] [instance: 620f0449-48c1-493a-a3c2-d741e032ba3e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 798.657032] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-80cfb065-a581-4f40-9311-a3271ce83134 tempest-ServerShowV254Test-185728470 tempest-ServerShowV254Test-185728470-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 798.657032] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-80cfb065-a581-4f40-9311-a3271ce83134 tempest-ServerShowV254Test-185728470 tempest-ServerShowV254Test-185728470-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 798.657739] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-80cfb065-a581-4f40-9311-a3271ce83134 tempest-ServerShowV254Test-185728470 tempest-ServerShowV254Test-185728470-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 798.661744] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-80cfb065-a581-4f40-9311-a3271ce83134 tempest-ServerShowV254Test-185728470 tempest-ServerShowV254Test-185728470-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 798.661744] nova-conductor[52243]: Traceback (most recent call last): [ 798.661744] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 798.661744] nova-conductor[52243]: return func(*args, **kwargs) [ 798.661744] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 798.661744] nova-conductor[52243]: selections = self._select_destinations( [ 798.661744] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 798.661744] nova-conductor[52243]: selections = self._schedule( [ 798.661744] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 798.661744] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 798.661744] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 798.661744] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 798.661744] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 798.661744] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 798.662319] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-80cfb065-a581-4f40-9311-a3271ce83134 tempest-ServerShowV254Test-185728470 tempest-ServerShowV254Test-185728470-project-member] [instance: 620f0449-48c1-493a-a3c2-d741e032ba3e] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager [None req-ac867706-159f-4836-9488-98276d914dbd tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 798.719488] nova-conductor[52244]: Traceback (most recent call last): [ 798.719488] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 798.719488] nova-conductor[52244]: return func(*args, **kwargs) [ 798.719488] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 798.719488] nova-conductor[52244]: selections = self._select_destinations( [ 798.719488] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 798.719488] nova-conductor[52244]: selections = self._schedule( [ 798.719488] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 798.719488] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 798.719488] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 798.719488] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 798.719488] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager [ 798.719488] nova-conductor[52244]: ERROR nova.conductor.manager [ 798.728105] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ac867706-159f-4836-9488-98276d914dbd tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 798.728105] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ac867706-159f-4836-9488-98276d914dbd tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 798.728105] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ac867706-159f-4836-9488-98276d914dbd tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 798.785064] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-ac867706-159f-4836-9488-98276d914dbd tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] [instance: 7e1b10c8-ead2-4c26-b8ff-d6f86f06b87d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 798.785880] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ac867706-159f-4836-9488-98276d914dbd tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 798.786333] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ac867706-159f-4836-9488-98276d914dbd tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 798.786547] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-ac867706-159f-4836-9488-98276d914dbd tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 798.791242] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-ac867706-159f-4836-9488-98276d914dbd tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 798.791242] nova-conductor[52244]: Traceback (most recent call last): [ 798.791242] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 798.791242] nova-conductor[52244]: return func(*args, **kwargs) [ 798.791242] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 798.791242] nova-conductor[52244]: selections = self._select_destinations( [ 798.791242] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 798.791242] nova-conductor[52244]: selections = self._schedule( [ 798.791242] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 798.791242] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 798.791242] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 798.791242] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 798.791242] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 798.791242] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 798.791832] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-ac867706-159f-4836-9488-98276d914dbd tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] [instance: 7e1b10c8-ead2-4c26-b8ff-d6f86f06b87d] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager [None req-4c13995c-bd38-4ec8-b67e-5af2ac70eb29 tempest-ServersTestManualDisk-900902045 tempest-ServersTestManualDisk-900902045-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 800.527978] nova-conductor[52243]: Traceback (most recent call last): [ 800.527978] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 800.527978] nova-conductor[52243]: return func(*args, **kwargs) [ 800.527978] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 800.527978] nova-conductor[52243]: selections = self._select_destinations( [ 800.527978] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 800.527978] nova-conductor[52243]: selections = self._schedule( [ 800.527978] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 800.527978] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 800.527978] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 800.527978] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 800.527978] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager [ 800.527978] nova-conductor[52243]: ERROR nova.conductor.manager [ 800.535122] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4c13995c-bd38-4ec8-b67e-5af2ac70eb29 tempest-ServersTestManualDisk-900902045 tempest-ServersTestManualDisk-900902045-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 800.536744] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4c13995c-bd38-4ec8-b67e-5af2ac70eb29 tempest-ServersTestManualDisk-900902045 tempest-ServersTestManualDisk-900902045-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 800.536744] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4c13995c-bd38-4ec8-b67e-5af2ac70eb29 tempest-ServersTestManualDisk-900902045 tempest-ServersTestManualDisk-900902045-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 800.599786] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-4c13995c-bd38-4ec8-b67e-5af2ac70eb29 tempest-ServersTestManualDisk-900902045 tempest-ServersTestManualDisk-900902045-project-member] [instance: 2d971f3f-6ea3-4ec4-809f-3e0609a3c576] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 800.600538] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4c13995c-bd38-4ec8-b67e-5af2ac70eb29 tempest-ServersTestManualDisk-900902045 tempest-ServersTestManualDisk-900902045-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 800.601283] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4c13995c-bd38-4ec8-b67e-5af2ac70eb29 tempest-ServersTestManualDisk-900902045 tempest-ServersTestManualDisk-900902045-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 800.601283] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-4c13995c-bd38-4ec8-b67e-5af2ac70eb29 tempest-ServersTestManualDisk-900902045 tempest-ServersTestManualDisk-900902045-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 800.605699] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-4c13995c-bd38-4ec8-b67e-5af2ac70eb29 tempest-ServersTestManualDisk-900902045 tempest-ServersTestManualDisk-900902045-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 800.605699] nova-conductor[52243]: Traceback (most recent call last): [ 800.605699] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 800.605699] nova-conductor[52243]: return func(*args, **kwargs) [ 800.605699] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 800.605699] nova-conductor[52243]: selections = self._select_destinations( [ 800.605699] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 800.605699] nova-conductor[52243]: selections = self._schedule( [ 800.605699] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 800.605699] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 800.605699] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 800.605699] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 800.605699] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 800.605699] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 800.605699] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-4c13995c-bd38-4ec8-b67e-5af2ac70eb29 tempest-ServersTestManualDisk-900902045 tempest-ServersTestManualDisk-900902045-project-member] [instance: 2d971f3f-6ea3-4ec4-809f-3e0609a3c576] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager [None req-629b0545-82b8-4b2d-9087-1be782ec0411 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 801.022940] nova-conductor[52244]: Traceback (most recent call last): [ 801.022940] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 801.022940] nova-conductor[52244]: return func(*args, **kwargs) [ 801.022940] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 801.022940] nova-conductor[52244]: selections = self._select_destinations( [ 801.022940] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 801.022940] nova-conductor[52244]: selections = self._schedule( [ 801.022940] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 801.022940] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 801.022940] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 801.022940] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 801.022940] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager [ 801.022940] nova-conductor[52244]: ERROR nova.conductor.manager [ 801.031424] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-629b0545-82b8-4b2d-9087-1be782ec0411 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 801.031685] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-629b0545-82b8-4b2d-9087-1be782ec0411 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 801.031854] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-629b0545-82b8-4b2d-9087-1be782ec0411 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 801.112795] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-629b0545-82b8-4b2d-9087-1be782ec0411 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] [instance: d76ca129-94f4-4d09-9be9-bb29fce459fd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 801.113539] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-629b0545-82b8-4b2d-9087-1be782ec0411 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 801.113772] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-629b0545-82b8-4b2d-9087-1be782ec0411 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 801.113944] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-629b0545-82b8-4b2d-9087-1be782ec0411 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 801.116998] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-629b0545-82b8-4b2d-9087-1be782ec0411 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 801.116998] nova-conductor[52244]: Traceback (most recent call last): [ 801.116998] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 801.116998] nova-conductor[52244]: return func(*args, **kwargs) [ 801.116998] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 801.116998] nova-conductor[52244]: selections = self._select_destinations( [ 801.116998] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 801.116998] nova-conductor[52244]: selections = self._schedule( [ 801.116998] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 801.116998] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 801.116998] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 801.116998] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 801.116998] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 801.116998] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 801.117523] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-629b0545-82b8-4b2d-9087-1be782ec0411 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] [instance: d76ca129-94f4-4d09-9be9-bb29fce459fd] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager [None req-149dd1ee-56f3-40da-9df3-c27288e71b32 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 803.206529] nova-conductor[52243]: Traceback (most recent call last): [ 803.206529] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 803.206529] nova-conductor[52243]: return func(*args, **kwargs) [ 803.206529] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 803.206529] nova-conductor[52243]: selections = self._select_destinations( [ 803.206529] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 803.206529] nova-conductor[52243]: selections = self._schedule( [ 803.206529] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 803.206529] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 803.206529] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 803.206529] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 803.206529] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager [ 803.206529] nova-conductor[52243]: ERROR nova.conductor.manager [ 803.215896] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-149dd1ee-56f3-40da-9df3-c27288e71b32 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 803.216209] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-149dd1ee-56f3-40da-9df3-c27288e71b32 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 803.216421] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-149dd1ee-56f3-40da-9df3-c27288e71b32 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 803.283295] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-149dd1ee-56f3-40da-9df3-c27288e71b32 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] [instance: 9f6a7260-2a24-40fe-8263-faef96b3800a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 803.285282] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-149dd1ee-56f3-40da-9df3-c27288e71b32 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 803.285502] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-149dd1ee-56f3-40da-9df3-c27288e71b32 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 803.285675] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-149dd1ee-56f3-40da-9df3-c27288e71b32 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 803.289526] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-149dd1ee-56f3-40da-9df3-c27288e71b32 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 803.289526] nova-conductor[52243]: Traceback (most recent call last): [ 803.289526] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 803.289526] nova-conductor[52243]: return func(*args, **kwargs) [ 803.289526] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 803.289526] nova-conductor[52243]: selections = self._select_destinations( [ 803.289526] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 803.289526] nova-conductor[52243]: selections = self._schedule( [ 803.289526] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 803.289526] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 803.289526] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 803.289526] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 803.289526] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 803.289526] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 803.292252] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-149dd1ee-56f3-40da-9df3-c27288e71b32 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] [instance: 9f6a7260-2a24-40fe-8263-faef96b3800a] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 804.099152] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-7708b3dc-3c0a-4916-88de-0c9dfdf12633 tempest-ServerGroupTestJSON-1094726720 tempest-ServerGroupTestJSON-1094726720-project-member] Acquiring lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 804.099152] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-7708b3dc-3c0a-4916-88de-0c9dfdf12633 tempest-ServerGroupTestJSON-1094726720 tempest-ServerGroupTestJSON-1094726720-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 804.099152] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-7708b3dc-3c0a-4916-88de-0c9dfdf12633 tempest-ServerGroupTestJSON-1094726720 tempest-ServerGroupTestJSON-1094726720-project-member] Lock "74f912f4-43d8-4d2a-9ea1-d6a83c370e35" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager [None req-7708b3dc-3c0a-4916-88de-0c9dfdf12633 tempest-ServerGroupTestJSON-1094726720 tempest-ServerGroupTestJSON-1094726720-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 804.193931] nova-conductor[52244]: Traceback (most recent call last): [ 804.193931] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 804.193931] nova-conductor[52244]: return func(*args, **kwargs) [ 804.193931] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 804.193931] nova-conductor[52244]: selections = self._select_destinations( [ 804.193931] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 804.193931] nova-conductor[52244]: selections = self._schedule( [ 804.193931] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 804.193931] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 804.193931] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 804.193931] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 804.193931] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager result = self.transport._send( [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager raise result [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager Traceback (most recent call last): [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._select_destinations( [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager selections = self._schedule( [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager [ 804.193931] nova-conductor[52244]: ERROR nova.conductor.manager [ 804.201685] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-7708b3dc-3c0a-4916-88de-0c9dfdf12633 tempest-ServerGroupTestJSON-1094726720 tempest-ServerGroupTestJSON-1094726720-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 804.201945] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-7708b3dc-3c0a-4916-88de-0c9dfdf12633 tempest-ServerGroupTestJSON-1094726720 tempest-ServerGroupTestJSON-1094726720-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 804.202182] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-7708b3dc-3c0a-4916-88de-0c9dfdf12633 tempest-ServerGroupTestJSON-1094726720 tempest-ServerGroupTestJSON-1094726720-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 804.248149] nova-conductor[52244]: DEBUG nova.conductor.manager [None req-7708b3dc-3c0a-4916-88de-0c9dfdf12633 tempest-ServerGroupTestJSON-1094726720 tempest-ServerGroupTestJSON-1094726720-project-member] [instance: 1f21a36e-f3b1-4f08-a9a7-eefabf614b89] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52244) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 804.248932] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-7708b3dc-3c0a-4916-88de-0c9dfdf12633 tempest-ServerGroupTestJSON-1094726720 tempest-ServerGroupTestJSON-1094726720-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 804.249205] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-7708b3dc-3c0a-4916-88de-0c9dfdf12633 tempest-ServerGroupTestJSON-1094726720 tempest-ServerGroupTestJSON-1094726720-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 804.249419] nova-conductor[52244]: DEBUG oslo_concurrency.lockutils [None req-7708b3dc-3c0a-4916-88de-0c9dfdf12633 tempest-ServerGroupTestJSON-1094726720 tempest-ServerGroupTestJSON-1094726720-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52244) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 804.252443] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-7708b3dc-3c0a-4916-88de-0c9dfdf12633 tempest-ServerGroupTestJSON-1094726720 tempest-ServerGroupTestJSON-1094726720-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 804.252443] nova-conductor[52244]: Traceback (most recent call last): [ 804.252443] nova-conductor[52244]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 804.252443] nova-conductor[52244]: return func(*args, **kwargs) [ 804.252443] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 804.252443] nova-conductor[52244]: selections = self._select_destinations( [ 804.252443] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 804.252443] nova-conductor[52244]: selections = self._schedule( [ 804.252443] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 804.252443] nova-conductor[52244]: self._ensure_sufficient_hosts( [ 804.252443] nova-conductor[52244]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 804.252443] nova-conductor[52244]: raise exception.NoValidHost(reason=reason) [ 804.252443] nova-conductor[52244]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 804.252443] nova-conductor[52244]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 804.253145] nova-conductor[52244]: WARNING nova.scheduler.utils [None req-7708b3dc-3c0a-4916-88de-0c9dfdf12633 tempest-ServerGroupTestJSON-1094726720 tempest-ServerGroupTestJSON-1094726720-project-member] [instance: 1f21a36e-f3b1-4f08-a9a7-eefabf614b89] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager [None req-0330af34-b970-401e-9a5f-179962f56644 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 804.868323] nova-conductor[52243]: Traceback (most recent call last): [ 804.868323] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 804.868323] nova-conductor[52243]: return func(*args, **kwargs) [ 804.868323] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 804.868323] nova-conductor[52243]: selections = self._select_destinations( [ 804.868323] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 804.868323] nova-conductor[52243]: selections = self._schedule( [ 804.868323] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 804.868323] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 804.868323] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 804.868323] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 804.868323] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1654, in schedule_and_build_instances [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 942, in _schedule_instances [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager result = self.transport._send( [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager raise result [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager Traceback (most recent call last): [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._select_destinations( [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager selections = self._schedule( [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager [ 804.868323] nova-conductor[52243]: ERROR nova.conductor.manager [ 804.874613] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0330af34-b970-401e-9a5f-179962f56644 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 804.874827] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0330af34-b970-401e-9a5f-179962f56644 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 804.874994] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0330af34-b970-401e-9a5f-179962f56644 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 804.908191] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-0330af34-b970-401e-9a5f-179962f56644 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] [instance: 4bdbceea-eb47-4fd3-baf5-8cb183574f2d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0fa786c9-f55e-46dc-b725-aa456ca9ff53',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52243) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1506}} [ 804.909177] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0330af34-b970-401e-9a5f-179962f56644 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 804.909177] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0330af34-b970-401e-9a5f-179962f56644 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 804.909281] nova-conductor[52243]: DEBUG oslo_concurrency.lockutils [None req-0330af34-b970-401e-9a5f-179962f56644 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52243) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 804.912088] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-0330af34-b970-401e-9a5f-179962f56644 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 804.912088] nova-conductor[52243]: Traceback (most recent call last): [ 804.912088] nova-conductor[52243]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 804.912088] nova-conductor[52243]: return func(*args, **kwargs) [ 804.912088] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 804.912088] nova-conductor[52243]: selections = self._select_destinations( [ 804.912088] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 804.912088] nova-conductor[52243]: selections = self._schedule( [ 804.912088] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 804.912088] nova-conductor[52243]: self._ensure_sufficient_hosts( [ 804.912088] nova-conductor[52243]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 804.912088] nova-conductor[52243]: raise exception.NoValidHost(reason=reason) [ 804.912088] nova-conductor[52243]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 804.912088] nova-conductor[52243]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 804.912676] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-0330af34-b970-401e-9a5f-179962f56644 tempest-ServerDiskConfigTestJSON-370129714 tempest-ServerDiskConfigTestJSON-370129714-project-member] [instance: 4bdbceea-eb47-4fd3-baf5-8cb183574f2d] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 849.623038] nova-conductor[52243]: ERROR nova.scheduler.utils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] [instance: a75a3491-94b0-4754-8e42-7bf49194a022] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2611, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2426, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2718, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance a75a3491-94b0-4754-8e42-7bf49194a022 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 849.623544] nova-conductor[52243]: DEBUG nova.conductor.manager [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Rescheduling: True {{(pid=52243) build_instances /opt/stack/nova/nova/conductor/manager.py:695}} [ 849.623799] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a75a3491-94b0-4754-8e42-7bf49194a022.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a75a3491-94b0-4754-8e42-7bf49194a022. [ 849.624030] nova-conductor[52243]: WARNING nova.scheduler.utils [None req-6480e2eb-733f-45d4-9040-d43f38b81be4 tempest-ServerShowV247Test-1842064909 tempest-ServerShowV247Test-1842064909-project-member] [instance: a75a3491-94b0-4754-8e42-7bf49194a022] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance a75a3491-94b0-4754-8e42-7bf49194a022.